Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

Gary Marcus Marcus on AI Published

TED, AI, and the test of time

We recorded 11 claims from this post. 9 were checked against evidence. 2 can’t be checked. 5 more are held. Read the post at its source.

Claims from this post (11)

In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.

  1. Assessment: UnresolvedSaid

    “I think it’s fair to say that the multimodal stuff still is very buggy, having troubles with basics like the English alphabet.”

    What we tested: OpenAI's multimodal capabilities remain very buggy as of April 2024, struggling even with basics like the English alphabet.

  2. Assessment: UnresolvedSaid

    “I am just not seeing that as OpenAI’s mission, anymore, especially when they have continued to fail to compensate most of the creators whose work they are drawing on, largely without consent.”

    What we tested: OpenAI no longer appears to pursue its mission of ensuring AGI benefits all of humanity, and it has continued not to compensate most creators whose work it uses, largely without their consent.

  3. Assessment: SupportedSaid

    “Yejin Choi worried that LLMs were not reliable. They still aren’t.”

    What we tested: LLMs remained unreliable as of April 2024, a year after Choi's TED talk.

    What happened: The March evaluation covers thirteen models from the Gemini, GPT, Claude and PaLM-2 families and documents continuing factuality limitations. It also finds better performance among larger models, so it supports continued unreliability without supporting stagnation. (arXiv; Jerry Wei and colleagues, )

  4. Assessment: SupportedSaid

    “I also urged the world to focus on global AI governance, and am pleased to see some genuine progress in that direction, especially led by the UN and the Council of Europe.”

    What we tested: There has been genuine progress on global AI governance, led especially by the UN and the Council of Europe.

    What happened: The UN's own account records adoption without a vote of an AI resolution addressing human rights and governance frameworks. Its broad international backing establishes a concrete multilateral milestone. (United Nations in Türkiye, )

  5. Assessment: UnresolvedSaid

    “I warned that LLMs would be used by bad actors, and they increasingly are.”

    What we tested: LLMs are increasingly being used by bad actors.

  6. Assessment: SupportedSaid

    “Eliezer Yudkowsky warned that AI was (inevitably!) going to kill us all. I am happy to report that so far it hasn’t.”

    What we tested: So far, AI has not killed humanity as Yudkowsky warned.

    What happened: The governing body's report documents living humans competing in the Boston Marathon on April 15. Their existence logically establishes that AI had not killed all humanity by then. (World Athletics, )

  7. Assessment: Can’t be checkedSaid

    “I love Khan Academy, but continue to have doubts about whether LLMs are really the right tool for the next step of the job.”

    What we tested: He continues to have doubts about whether LLMs are really the right tool for the next step of the job of transforming education.

  8. Assessment: SupportedSaid

    “The Humane AI pin/virtual assistant just doesn’t work well, and a lot of the problem is the underlying AI, which I believe is powered by OpenAI.”

    What we tested: The Humane AI pin just doesn't work well, and a lot of the problem is its underlying AI, which he believes is powered by OpenAI.

    What happened: The launch review documents a false answer about a California food ban and a mistaken temple identification. These failures directly impair the question-answering function, although the review also identifies missing features and hardware problems. (WIRED, )

  9. Assessment: UnresolvedSaid

    “As I put it to the Times, no company yet had A.I. technology that was sophisticated enough to make a virtual assistant answer questions reliably,”

    What we tested: As of April 2024, no company has AI technology sophisticated enough to make a virtual assistant answer questions reliably.

  10. Assessment: Can’t be checkedSaid

    “I don’t think we are any closer to a machine with the wherewithal or the motivation to annihilate the human species than we were a year ago.”

    What we tested: He does not think we are any closer to a machine with the wherewithal or the motivation to annihilate the human species than we were a year ago.

  11. Assessment: SupportedSaid

    “Adding visual data to LLMs hasn’t solved hallucinations.”

    What we tested: Adding visual data to LLMs has not solved hallucinations.

    What happened: The developer of GPT-4V, the main model that added vision to an LLM, reports that it was still prone to hallucinations. (OpenAI, )

Held from this post (5)

These records did not pass every check, so nothing about them is published. Held is not a finding.