Race, statistics, and the persistent cognitive limitations of DALL-E
We recorded 7 claims from this post. 2 were checked against evidence. 5 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (7)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“What one really needs, and no one yet knows how to build, is a system that can derive semantics of wholes from their parts as a function of their syntax.”
What we tested: AI needs systems that derive the meaning of wholes from their parts according to syntax, which no one yet knows how to build.
Assessment: UnresolvedSaid
“About 18 months have passed; although there are certainly improvements (the images are vastly better), owing presumably in large part to a larger training set, the fundamental approach (mapping words to images, rather than constructing intermediate models of the world and how language and images relate to those models) remains the same, and so the core issue remains.”
What we tested: About 18 months after DALL-E 2, the images are vastly better, presumably largely because of a larger training set, but the fundamental approach of mapping words to images rather than building intermediate world models is the same, and so the core compositionality issue remains.
Assessment: Can’t be checkedSaid
“It’s that it can’t separate the world from the statistics of its dataset. DALL-E does not have a cognitive construct of a doctor or a patient or a human being or an occupation or medicine or race or egalitarianism or equal opportunity or any of that.”
What we tested: DALL-E 3 cannot distinguish the world from its training data statistics and has no cognitive concepts of doctors, patients, race, or similar.
Assessment: UnresolvedSaid
“It’s the same issue of canonical versus noncanonical statistics, all over again.”
What we tested: Multimodal ChatGPT's trouble telling time from watch images comes from the same canonical-versus-noncanonical statistics problem seen in DALL-E.
Assessment: Can’t be checkedSaid
“Blind fealty to the statistics of arbitrary data is not AGI. It’s a hack. And not one that we should want.”
What we tested: Blind fealty to the statistics of arbitrary data is not AGI; it is a hack, and not one we should want.
Assessment: Can’t be checkedSaid
“Our focus should be on figuring on how to build AI that can represent and reason about *values*, rather than simply perpetuating past data”.”
What we tested: AI research should focus on building systems that can represent and reason about values instead of just perpetuating past data.
Assessment: Can’t be checkedSaid
“But the level of actual cognitive comprehension of here is low. The modest number of properly labeled examples is presumably swamped by all the billions of watch advertisements.”
What we tested: Multimodal ChatGPT has low real comprehension, and the few correctly labeled clock examples are probably drowned out by billions of watch ads showing the canonical time.
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.