What was 60 Minutes thinking, in that interview with Geoff Hinton?
We recorded 20 claims from this post. 12 were checked against evidence. 8 can’t be checked. 9 more are held. Read the post at its source.
Claims from this post (20)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
Observation
“When it comes to AI in particular, we are getting way ahead of our skis, rushing forward a technology we don’t fully understand.”
Our reading: Humanity is rushing forward with AI, a technology it does not fully understand.
No horizon Evidence as of
Limitation: Support applies to deployment proceeding despite incomplete understanding.
Assessment: Can’t be checkedSaid
Observation
“But I don’t we are all that close to machines that are more intelligent than us”
Our reading: We are not very close to machines that are more intelligent than humans.
No horizon
Qualifier: partly definitional
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Since they can’t reliably solve them, I don’t think we should attribute “understanding” to them, at least not in any remotely deep sense of the word understanding.”
Our reading: We should not attribute deep understanding to generative AI systems because they cannot reliably solve basic problems like family-relationship inference.
No horizon
Qualifier: because they cannot reliably solve basic inference problems
Assessment: Can’t be checkedSaid
Observation
“Of course, I don’t think these machines are conscious, either.”
Our reading: Current AI machines are not conscious.
No horizon
Assessment: UnresolvedSaid
Observation
“I think it’s fair to say that generative AI works better than most people expected.”
Our reading: Generative AI works better than most people expected.
No horizon Evidence as of
Limitation: A narrower reading about informed AI researchers could produce a different assessment.
Assessment: SupportedSaid
Observation
“Actually … a lot of us still think you are declaring victory prematurely.”
Our reading: A lot of people, Marcus included, still think Hinton is declaring victory for neural networks prematurely.
No horizon Evidence as of
Limitation: The result depends on reading 'premature victory' as unresolved scientific problems rather than an explicit response to Hinton.
Assessment: SupportedSaid
Observation
“There’s some information in there, but whatever’s there is often both patchy and problematic.”
Our reading: Chatbots contain some information, but whatever is there is often both patchy and problematic.
No horizon Evidence as of
Limitation: Biography and family-relationship tasks are not representative of every chatbot use.
Assessment: SupportedSaid
Observation
“The connections in chatbots are very different from the connections in the brain; it’s a mistake to compare them directly in this way.”
Our reading: The connections in chatbots are very different from those in the brain, so comparing them directly in the way Hinton did (by counting connections) is a mistake.
No horizon Evidence as of
Limitation: The evidence does not establish which system learns better on a specified task.
Assessment: MixedSaid
Causal claim
“It makes current AI hard to debug (nobody knows for example how to ground them in facts), and it makes them difficult predict, which means, unlike calculators or spreadsheets, we don’t really know what’s going to happen when we ask them a question.”
Our reading: Because we don't really understand how trained neural networks do what they do, current AI is hard to debug (for example, nobody knows how to ground them in facts) and difficult to predict, so unlike calculators or spreadsheets we don't really know what will happen when we ask them a question.
No horizon Evidence as of
Limitation: The supplied Anthropic passage describes interpretability obstacles, rather than directly measuring difficulty diagnosing or fixing failures; its demonstrated progress concerns a small model [e1].
Assessment: SupportedSaid
Observation
“Just like we can’t guarantee that they won’t make stuff up, we don’t know how to guarantee that they won’t write flawed code.”
Our reading: We cannot guarantee that AI systems will not fabricate or write flawed code.
No horizon Evidence as of
Limitation: This is an empirical assessment of contemporary general-purpose systems, not a mathematical impossibility claim.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We are giving way too much authority to machines that we can’t control.”
Our reading: We are giving way too much authority to machines that we can't control.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“this is one of the reasons laws should be written to make machines disclose the fact that they are machines.”
Our reading: Laws should require machines to disclose that they are machines.
No horizon
Assessment: UnresolvedSaid
Observation
“Your best example of a spectacular machine invention is in fact a perfect example of the kind of parroting and theft of intellectual property that is characteristic of large language models.”
Our reading: Bard's 'baby shoes' story was parroting of an existing story, and such parroting and IP theft is characteristic of large language models.
No horizon Evidence as of
Limitation: The available transcript is an updated version, and the independent transcription contains only part of the output.
Assessment: Can’t be checkedSaid
Observation
“To be fair, large language models aren’t just looking things up, but the idea that a good prediction of next word necessarily implies understanding is fallacious.”
Our reading: Good next-word prediction by LLMs does not necessarily imply understanding.
No horizon Evidence as of
Limitation: This does not reject Marcus's argument or endorse Hinton's.
Assessment: UnresolvedSaid
Observation
“You know how many radiologists have been replaced by machines seven years later? Zero.”
Our reading: Seven years after Hinton's 2016 prediction, no radiologists have been replaced by machines.
No horizon Evidence as of
Limitation: The workforce census covers the UK, while the claim supplies no geographic limit.
Assessment: UnresolvedSaid
Forecast
“though I agree with Geoff that eventually AI will have a big impact on drug design. Perhaps with current techniques, perhaps not; we will have to see.”
Our reading: AI will eventually have a big impact on drug design, possibly but not necessarily with current techniques.
No deadline stated Evidence as of
Limitation: The threshold for substantial impact is an evaluator interpretation.
Assessment: UnresolvedSaid
Forecast
“And emphasize that wholesale, automated fake news will be used both to manipulate markets and elections, and might undermine democracy.”
Our reading: Wholesale, automated fake news will be used to manipulate both markets and elections, and it might undermine democracy.
No deadline stated Evidence as of
Limitation: Wholesale and automated require interpretation.
Assessment: UnresolvedSaid
Causal claim
“I can’t either; there’s a lot we can do to help, but nothing I can see either to absolutely guarantee safety. Rushing ahead is creating risk.”
Our reading: A lot can be done to help with AI safety, but Marcus can see nothing that would absolutely guarantee it, and rushing ahead is creating risk.
No horizon Evidence as of
Limitation: The company reports do not isolate the effect of development or deployment speed [e1, e2].
Assessment: Can’t be checkedSaid
Observation
“I am much more worried about bad actors deliberately misusing AI, than machines deliberately wanting to take over. But Geoff’s right that we can’t fully rule it out either.”
Our reading: Marcus is much more worried about bad actors deliberately misusing AI than about machines deliberately wanting to take over, though the possibility of a takeover cannot be fully ruled out.
No horizon
Qualifier: cannot fully rule it out
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“but we are in complete agreement that we are at a turning point with enormous uncertainty, and that we need to make the right choices now.”
Our reading: AI is at a turning point of enormous uncertainty, and society needs to make the right choices now.
No horizon
Held from this post (9)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: AI does not grasp the big picture as human radiologists do; it is better at vision than at understanding case files.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: The unpredictability of current AI makes engineering with it exceptionally hard and is one reason some companies are cautious about using it.
- The criteria are ambiguous. Our reading, not assessed: The AI systems Hinton says 'work well' still cannot be trusted, make random mistakes, and have no basis in factuality, so calling them 'working well' is tendentious.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: In medical imaging, the best results still come from combining machine vision with human understanding.
- The criteria are ambiguous. Our reading, not assessed: Current generative AI systems do not really understand what they say and cannot reliably reason flexibly about novel things.
- The record does not meet the current rules. Our reading, not assessed: As far as Marcus knows, AI has so far produced only candidate drugs, none yet proven to work.