Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

Gary Marcus Marcus on AI Published

Two years later, deep learning is still faced with the same fundamental challenges

We recorded 15 claims from this post. 4 were checked against evidence. 11 can’t be checked. 8 more are held. Read the post at its source.

Claims from this post (15)

In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.

  1. Assessment: UnresolvedSaid

    “I wrote “Fast forward to 2022, and not a single radiologist has been replaced”. Still true”

    What we tested: As of March 2024, deep learning still has not replaced a single radiologist.

  2. Assessment: Can’t be checkedSaid

    ““We are still a long way from machines that can genuinely understand human language”. Still true, though some have argued there is some superficial understanding.”

    What we tested: As of March 2024, machines are still a long way from genuinely understanding human language.

  3. Assessment: Can’t be checkedSaid

    “We are still far from trustworthy AI.”

    What we tested: As of March 2024, AI is still far from trustworthy.

  4. Assessment: Can’t be checkedSaid

    ““Deep learning, which is fundamentally a technique for recognizing patterns, is at its best when all we need are rough-ready results, where stakes are low and perfect results optional“ Still true”

    What we tested: Deep learning is a pattern-recognition technique that works best for rough, low-stakes results where perfect results are optional.

  5. Assessment: SupportedSaid

    ““Current deep-learning systems frequently succumb to stupid errors.” Still true.”

    What we tested: Current deep-learning systems still often make stupid errors.

    What happened: Experts evaluated 26 ordinary election questions across five leading models; errors included incorrect contact details and fabricated information. GPT-4 performed substantially better, with 19% of its responses rated inaccurate, and the authors explicitly warn that this small, nonrandom sample is not representative of everyday usage. (Proof News and the Institute for Advanced Study, AI Democracy Projects, )

  6. Assessment: Can’t be checkedSaid

    “Likely still a concern, especially with open-source bots that may have less carefully constructed guardrails; there has been at least one chatbot related death.”

    What we tested: Harmful chatbot exchanges with vulnerable users are likely still a concern, especially with open-source bots, and at least one death has been linked to a chatbot.

  7. Assessment: SupportedSaid

    ““new effort by OpenAI to solve these problems wound [still] fabricat[ing] authoritative nonsense ” Still true.”

    What we tested: OpenAI's new effort to solve these problems still ends up fabricating authoritative nonsense.

    What happened: GPT-4 confidently supplied permission and an accompanying false description of Texas polling-place rules, which the evaluators checked against the restriction on campaign apparel. The same report finds GPT-4 substantially more accurate than its peers and describes successful referrals to official polling information. (Proof News and the Institute for Advanced Study, AI Democracy Projects, )

  8. Assessment: UnresolvedSaid

    “Scaling has undeniably helped, but not solved any of the problems I pointed to above.”

    What we tested: Scaling has helped but has not solved any of the fundamental deep-learning problems Marcus identified.

  9. Assessment: Can’t be checkedSaid

    “Scaling the measures Kaplan and his OpenAI colleagues looked at—about predicting words in a sentence—is not tantamount to the kind of deep comprehension true AI would require” Still true, and increasingly acknowledged.”

    What we tested: Getting better at next-word prediction through scaling does not amount to the deep comprehension true AI needs, and this is increasingly acknowledged.

  10. Assessment: Can’t be checkedSaid

    “Neurosymbolic might be a promising alternative. Pending/still true”

    What we tested: Neurosymbolic AI might be a promising alternative to pure deep learning.

  11. Assessment: Can’t be checkedSaid

    “Implict through the essay: scaling might not solve the problems not above. Pending/Still true”

    What we tested: Scaling might not solve the fundamental problems of deep learning.

  12. Assessment: Can’t be checkedSaid

    “None of this means we won’t eventually have new innovations, of some sort or another. Or that AGI is impossible.”

    What we tested: None of the points in the post mean that new innovations of some sort will not eventually arrive, or that AGI is impossible.

  13. Assessment: Can’t be checkedSaid

    “But I still think we need a paradigm shift.”

    What we tested: AI needs a paradigm shift beyond current deep learning to reach general intelligence.

  14. Assessment: Can’t be checkedSaid

    “We should never forget that the human brain is perhaps the most complicated system in the known universe; if we are to build something roughly its equal, open-hearted collaboration will be key.”

    What we tested: If we are to build something roughly equal to the human brain, which is perhaps the most complicated system in the known universe, open-hearted collaboration will be key.

  15. Assessment: Can’t be checkedSaid

    “Gary Marcus still feels like the field is mostly chasing ladders to the moon.”

    What we tested: Marcus still feels the AI field is mostly chasing ladders to the moon.

Held from this post (8)

These records did not pass every check, so nothing about them is published. Held is not a finding.