Sora still appears to have trouble with physics
We recorded 6 claims from this post. 5 were checked against evidence. 1 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (6)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
“In February, the day after Sora came out, I warned that it didn’t really understand physics—and wouldn’t — in essay called Sora’s Surreal Physics:”
What we tested: Sora does not really understand physics and will not come to understand it.
Assessment: UnresolvedSaid
“Sora is never really getting it, even with mountains and mountains of data.”
What we tested: Sora will never really grasp object permanence, even with vast amounts of data.
Assessment: SupportedSaid
“That gross violation of spatiotemporal continuity/failure of object permanence is not a one-off, either; it’s something general.”
What we tested: Sora's failures of object permanence and spatiotemporal continuity are general rather than isolated.
What happened: The developer identifies spontaneous appearance as a model weakness associated particularly with crowded scenes. This supports recurrence as a failure mode, without quantifying its prevalence. (OpenAI, )
Assessment: UnresolvedSaid
“And like other generative AI systems, there is no way to encode (and guarantee) constraints like “be truthful” or “obey the laws of physics”or “don’t just invent (or eliminate) objects”.”
What we tested: Sora and other generative AI systems offer no way to encode and guarantee constraints such as truthfulness, obeying physics, or not inventing or eliminating objects.
Assessment: UnresolvedSaid
“More data won’t solve [Sora’s] problem [with physics].”
What we tested: More training data will not solve Sora's problems with physics.
Assessment: Can’t be checkedSaid
“The John Locke hypothesis that you can learn physics purely from sense data is failing, over, and over, and over again; this time perhaps (if I had to guess) to the tune of a billion dollars.”
What we tested: The author guesses that Sora's failed attempt to learn physics from data may have cost around a billion dollars.
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: The December 2024 release of Sora appears to still have the physics and object-permanence problems seen in February.
- The two reviews read the claim differently