Claims that can’t be checked by Gary Marcus, page 12 of 24

Claims that can’t be checked by Gary Marcus (2370)

Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.

Page 12 of 24: claims 1101 to 1200 of 2370, ordered by the date first said, newest first. To find a claim by its words, use search.

1. Assessment: Can’t be checkedSaid 2025-01-26

   “One that deserves Congressional investigation was Meta's decision to open source their LLMs.”

2. Assessment: Can’t be checkedSaid 2025-01-26

   “(Trump’s boosting of Stargate seemingly fits with the same magical thinking in LLMs, premised on the same hope of achieving a supremacy that may never come).”

3. Assessment: Can’t be checkedSaid 2025-01-26

   “Altman may be a master salesman, but Musk is correct that the US should not be so reliant on him and should not have given him such an eminent seal of approval based on so little.”

4. Assessment: Can’t be checkedSaid 2025-01-26

   “I have said it before, and I will say it again—OpenAI may well become the WeWork of AI.”

5. Assessment: Can’t be checkedSaid 2025-01-26

   “The race to AGI will be won not by the country with the most chips but by the one that best fosters true innovation.”

6. Assessment: Can’t be checkedSaid 2025-01-26

   “Others may catch up, too, because LLMs just got a lot cheaper, and consequently the requirement for vast arrays of special purpose hardware has somewhat diminished.”

7. Assessment: Can’t be checkedSaid 2025-01-26

   “If all this is true, the CHIPS Act, which was designed to slow China in the AI race, may turn out to be one of the worst backfires in history.”

8. Assessment: Can’t be checkedSaid 2025-01-26

   “If the race to LLMs continues to be unregulated in the US, and LLMs remain detached from reality, rushed development cycles and intense global competition might well exacerbate risks of misinformation, biased outputs, privacy breaches, and misuse by malicious actors.”

9. Assessment: Can’t be checkedSaid 2025-01-26

   “Building $500B worth of power and data centers in the service of enormous collections of those chips isn't looking so sensible, either.”

10. Assessment: Can’t be checkedSaid 2025-01-26

    “Furthermore, as I have argued for years, LLMs are too opaque, too unwieldy, and too difficult to debug and verify. The answer lies elsewhere; betting our future on that single idea is foolish.”

11. Assessment: Can’t be checkedSaid 2025-01-26

    “And maybe kneecapped our greatest silicon company, Nvidia.”

12. Assessment: Can’t be checkedSaid 2025-01-26

    “Nvidia could soon take a serious hit, too, for two reasons. First, the DeepSeek results suggest that large language models might be trained far more efficiently going forward.”

13. Assessment: Can’t be checkedSaid 2025-01-22

    “Like a lot of what Sam says, this is based on the conjecture, or in this case multiple conjectures:”

14. Assessment: Can’t be checkedSaid 2025-01-22

    “The equally speculative conjecture that the U.S. will long term have an advantage over China if the U.S. builds the hypothesized infrastructure, as if China isn’t trying to build the same.”

15. Assessment: Can’t be checkedSaid 2025-01-22

    “The entirely speculative conjecture that LLMs or something else OpenAI figures out how to be build will be enormously profitable.”

16. Assessment: Can’t be checkedSaid 2025-01-22

    “Elon thinks Sam is playing the same move that they once played together. He could be right.”

17. Assessment: Can’t be checkedSaid 2025-01-22

    “The entirely speculative conjecture that any profits will actually do much to help the American people, as opposed to just enriching those who own that infrastructure.”

18. Assessment: Can’t be checkedSaid 2025-01-20

    “Sam Altman himself hinted we were close, just two weeks ago, writing “We are now confident we know how to build AGI as we have traditionally understood it”, which to be honest gave me Elizabeth Holmes vibes.”

19. Assessment: Can’t be checkedSaid 2025-01-15

    “But not this year (or next, or the one after that, and probably not this decade, except in narrow use cases).”

20. Assessment: Can’t be checkedSaid 2025-01-13

    “A system answering a river-crossing problem should not answer by superficial analogy to the words in similar problems found in a giant scrape of the internet, but rather by reasoning through concepts about boats, rivers, people, and so on.”

21. Assessment: Can’t be checkedSaid 2025-01-13

    “Anyone who tries to tell you otherwise is either confused or trying to hype something.”

22. Assessment: Can’t be checkedSaid 2025-01-13

    “but that regurgitation is an illusion. Genuine depth is lacking.”

23. Assessment: Can’t be checkedSaid 2025-01-13

    “Moving past broad but shallow AI and into a new and hopefully safer and more reliable regime should be a major research priority for the human species.”

24. Assessment: Can’t be checkedSaid 2025-01-13

    “Counting on AI that is shallow and not reliable, it should also be said, could cause huge problems.”

25. Assessment: Can’t be checkedSaid 2025-01-13

    “Then, and only then, will we have reliable solutions to the unreliability that has plagued recent AI.”

26. Assessment: Can’t be checkedSaid 2025-01-13

    “For all these reasons, I propose that we call LLMs broad, shallow intelligence..”

27. Assessment: Can’t be checkedSaid 2025-01-13

    “Instead, LLMs are still unpredictable enough that humans should still be in the loop in most case.”

28. Assessment: Can’t be checkedSaid 2025-01-13

    “We can never guarantee that broad but shallow AI will be safe, or that it will be human-compatible (in the words of Stuart Russell), or that it will do what we ask of it.”

29. Assessment: Can’t be checkedSaid 2025-01-13

    “Every declaration of us already reaching AGI I have seen rests on redefining the term downwards, as I discussed last week,”

30. Assessment: Can’t be checkedSaid 2025-01-13

    “Nobody serious thinks that we have reached AGI (artificial general intelligence) yet.”

31. Assessment: Can’t be checkedSaid 2025-01-13

    “The river crossing example and many others shows that LLMs often use the words without a deep understanding what those words mean.”

32. Assessment: Can’t be checkedSaid 2025-01-09

    “o3 is a real advance in some domains beyond GPT-4, but it’s likely not the quantum leap forward people would expect of the name GPT-5, and at a rumored $1000/querry certainly not at the price point anyone might have hoped for”

33. Assessment: Can’t be checkedSaid 2025-01-09

    “Gary Marcus wishes the media would hold those who make unrealistic promises to account.”

34. Assessment: Can’t be checkedSaid 2025-01-09

    “We might see GPT-5 this year, but given bottlenecks on data and inherent limits in LLMs we may well not.”

35. Assessment: Can’t be checkedSaid 2025-01-09

    “He made some of his usual overpromises (500,000 humanoid robots in three years, which seems to very unlikely)”

36. Assessment: Can’t be checkedSaid 2025-01-09

    “I won’t be at all surprised if don’t even see anything worthy of the GPT-5 name”

37. Assessment: Can’t be checkedSaid 2025-01-07

    “There are some applications where the current systems are reliable enough to be practically useful, though not nearly as many as is often claimed.”

38. Assessment: Can’t be checkedSaid 2025-01-07

    “If a lot of people’s goalposts have moved, mine have not. same as before, I will be satisfied if and when we reach AI systems are “flexible and general, with resourcefulness and reliability comparable to (or beyond) human intelligence” – and not until.”

39. Assessment: Can’t be checkedSaid 2025-01-07

    “In reality, the definition of AGI, while not perfect, is reasonably clear — and has been reasonably clear — for a long time.”

40. Assessment: Can’t be checkedSaid 2025-01-07

    “Current models trained on next-token prediction have remarkable abilities and remarkable weaknesses. The scope of those abilities and weaknesses of the current systems is not well understood.”

41. Assessment: Can’t be checkedSaid 2025-01-06

    “But so far I have not seen a principled solution to even one of these longstanding problems, and that tells me a lot.”

42. Assessment: Can’t be checkedSaid 2025-01-06

    “[Note also that the current idea, test-time “scaling”, is very expensive, and will likely prove unreliable out of domains where augmentation is adequate, in which case it will get comparatively little commercial traction.”

43. Assessment: Can’t be checkedSaid 2025-01-06

    “In domains where that’s not possible, o1 may not even be that much better than GPT-4 (e.g on some language tasks), as you can see if you read OpenAI’s reports carefully.”

44. Assessment: Can’t be checkedSaid 2025-01-06

    “o1 probably isn’t as robust as people might have hoped. Even on certain benchmarks, results may be fragile or difficult to replicate:”

45. Assessment: Can’t be checkedSaid 2025-01-06

    “In o1’s best performances cases, there may have been heavy data augmentation that is not possible in more open-ended domains.”

46. Assessment: Can’t be checkedSaid 2025-01-06

    “Here is a thread of links to a several recent observations that I think show how far we are from robust intelligence.”

47. Assessment: Can’t be checkedSaid 2025-01-06

    “I do not share his confidence.”

48. Assessment: Can’t be checkedSaid 2025-01-06

    “The presentation of o3 was problematic, and no clean test of out-of-domain generalization was provided, suggesting that we may see the kinds of problems with distribution shift as ever, outside of semi-closed domains where synthetic data can readily be generated.”

49. Assessment: Can’t be checkedSaid 2025-01-05

    “But fundamentally its videos are more like visual collages than the renderings of game engines, which is why they sometimes have a surreal or magic trick quality.”

50. Assessment: Can’t be checkedSaid 2025-01-05

    “In our view, it is only once AI researchers grapple directly with the challenging problems inherent in commonsense reasoning about entities that interact in space and persist in time that the field of general-purpose AI will finally begin to mature.”

51. Assessment: Can’t be checkedSaid 2025-01-03

    “My essay was about the small but very real possibility that 2025 could see the worst cyberattack in history, perhaps catalyzed in some fashion by generative AI (which was #24 in my list of 25 AI predictions for 2025).”

52. Assessment: Can’t be checkedSaid 2025-01-01

    “Technical moat will continue to be elusive. Instead, there will be more convergence on broadly similar models, across both US and China; some systems in Europe will catch up to roughly the same place.”

53. Assessment: Can’t be checkedSaid 2025-01-01

    “Few companies (and even fewer consumers) will adopt o3 at wide scale because of concerns about price and robustness relative to that price.”

54. Assessment: Can’t be checkedSaid 2025-01-01

    “Sora will continue to have trouble with physics.”

55. Assessment: Can’t be checkedSaid 2025-01-01

    “My predictions, going back to 2022 that pure scaling of LLMs would eventually start to run out appear to have been confirmed in November and December, based on comments from many industry insiders.”

56. Assessment: Can’t be checkedSaid 2025-01-01

    “2025 could well be the year in which valuations for major AI companies start to fall.”

57. Assessment: Can’t be checkedSaid 2025-01-01

    “Companies will continue to experiment with AI, but adoption to production-grade systems scaled out in the real-world will continue to be tentative.”

58. Assessment: Can’t be checkedSaid 2025-01-01

    “Instead we may see models like o1 that are quite good at many tasks for which high-quality synthetic data can be created, but in other domains only incrementally better than GPT-4.”

59. Assessment: Can’t be checkedSaid 2025-01-01

    “My 2023 prediction that OpenAI might someday be seen as the WeWork of AI has not been confirmed, but they are far from out of the woods.”

60. Assessment: Can’t be checkedSaid 2025-01-01

    “Sora will continue to have trouble with physics.”

61. Assessment: Can’t be checkedSaid 2025-01-01

    “Few companies (and even fewer consumers) will adopt o3 at wide scale because of concerns about price and robustness relative to that price.”

62. Assessment: Can’t be checkedSaid 2025-01-01

    “Long term security, especially at high valuations, is this far from assured.”

63. Assessment: Can’t be checkedSaid 2025-01-01

    “Technical moat will continue to be elusive. Instead, there will be more convergence on broadly similar models, across both US and China; some systems in Europe will catch up to roughly the same place.”

64. Assessment: Can’t be checkedSaid 2025-01-01

    “Neurosymbolic AI will become much more prominent.”

65. Assessment: Can’t be checkedSaid 2025-01-01

    “o3 has been previewed but not released likely will likely be quite expensive.)”

66. Assessment: Can’t be checkedSaid 2025-01-01

    “We may well see a large-scale cyberattack in which Generative AI plays an important causal role, perhaps in one of the four ways discussed in a short essay of mine that will appear shortly in Politico.”

67. Assessment: Can’t be checkedSaid 2025-01-01

    “Commercial artists and voiceover actors have perhaps been the hardest hit so far.”

68. Assessment: Can’t be checkedSaid 2025-01-01

    “I wouldn’t be shocked if none were reliably solved by the end of the year.”

69. Assessment: Can’t be checkedSaid 2025-01-01

    “(Google’s Veo 2 seems to be better but I have not been able to experiment with it, and suspect that changes of state and the persistence of objects will still cause problems;”

70. Assessment: Can’t be checkedSaid 2025-01-01

    “There could continue to be no “GPT-5 level” model (meaning a huge, across the board quantum leap forward as judged by community consensus) throughout 2025.”

71. Assessment: Can’t be checkedSaid 2025-01-01

    “AI will presumably solve all of these someday, of course, but current techniques have thus far not been adequate.”

72. Assessment: Can’t be checkedSaid 2024-12-30

    “Both of us agree that the achievement of 8 out of 10 of these tasks would be strong evidence for the G (General) in Artificial General Intelligence having been achieved.”

73. Assessment: Can’t be checkedSaid 2024-12-22

    “Because of the pretraining, and lack of comparability, what was actually shown was disappointing.”

74. Assessment: Can’t be checkedSaid 2024-12-22

    “The media should be asking hard questions, not fanning hype.”

75. Assessment: Can’t be checkedSaid 2024-12-22

    “Until there is considerable external scientific scrutiny (so far there has been none), we won’t really know exactly what the o3 advance is or how important it is.”

76. Assessment: Can’t be checkedSaid 2024-12-22

    “As the scientist Adan Becerra, PhD put it (and Chollet publicly agreed) the best thing would have been to present data for the “base model” without the pretraining.”

77. Assessment: Can’t be checkedSaid 2024-12-22

    “The video should have been much clearer about what was actually tested and what was actually trained.”

78. Assessment: Can’t be checkedSaid 2024-12-22

    “As NYU prof Brenden Lake pointed out, the test should never have been called ARC-AGI.”

79. Assessment: Can’t be checkedSaid 2024-12-22

    “People in the media probably shouldn’t even joke about o3 being AGI.”

80. Assessment: Can’t be checkedSaid 2024-12-21

    “Spending a quarter trillion dollars in order to have something 1000 times more expensive than average humans is maybe not the way to go.”

81. Assessment: Can’t be checkedSaid 2024-12-21

    “Until lots of people get to try o3 on many different kinds of tasks, we should not assume that it is reliable.”

82. Assessment: Can’t be checkedSaid 2024-12-21

    “I wish that others would give lines in the sand in advance; instead, they are likely to craft definitions post hoc, or not give them at all.”

83. Assessment: Can’t be checkedSaid 2024-12-21

    “And I am saying, as every roboticist on the planet knows, that we shouldn’t take demos seriously, until purported products are actually released and subjected to outside scrutiny.”

84. Assessment: Can’t be checkedSaid 2024-12-21

    “Furthermore, the impact of all this on the environment could be immense.”

85. Assessment: Can’t be checkedSaid 2024-12-21

    “As in O3 may work for problems where massive data augmentation is possible, but not in others where it’s not.”

86. Assessment: Can’t be checkedSaid 2024-12-21

    “I couldn’t agree more. Yesterday’s demo, impressive as it was, doesn’t change that.”

87. Assessment: Can’t be checkedSaid 2024-12-21

    “Nothing in the o3 presentation yesterday suggested to me that this decades-old core problem has been remotely solved.”

88. Assessment: Can’t be checkedSaid 2024-12-14

    “The problems by now seem clear. Suchir was right to consider them.”

89. Assessment: Can’t be checkedSaid 2024-12-14

    “Society needs to do more to protect people like Suchir Balaji.”

90. Assessment: Can’t be checkedSaid 2024-12-14

    “It is unfortunate that CA SB-1047’s whistleblower protections were vetoed.”

91. Assessment: Can’t be checkedSaid 2024-12-12

    “I wouldn’t take seriously any of its anthropomorphic language about LLMs “knowing” or having scheming or having intentions to deceive, etc.”

92. Assessment: Can’t be checkedSaid 2024-12-12

    “They more power we give them (but hooking up them to agents, people, etc), the more risk we are incurring.”

93. Assessment: Can’t be checkedSaid 2024-12-12

    “Even if machines don’t ever rise up a la the Terminator, a bad actor could still leverage LLMs to cause massive mayhem. Possibly soon.”

94. Assessment: Can’t be checkedSaid 2024-12-12

    “The important thing is that paper makes it very clear that nobody should ever take LLMs at their word.”

95. Assessment: Can’t be checkedSaid 2024-12-12

    “The longer we delay in regulation, the more risk we are incurring.”

96. Assessment: Can’t be checkedSaid 2024-12-12

    “There’s a lot of anthropomorphism here in this paper that I find hard to take, but it is in other ways a smart and carefully considered paper — and ought to be a wakeup call.”

97. Assessment: Can’t be checkedSaid 2024-12-12

    “That may leave all of us extremely vulnerable to this mess.”

98. Assessment: Can’t be checkedSaid 2024-12-11

    “I don’t doubt this, when stated so vaguely, but you can’t improve the engineering if you can’t identify the problems and weak points, such that you can address them.”

99. Assessment: Can’t be checkedSaid 2024-12-11

    “But ignoring all the evidence that suggests that we won’t be there in the very near term does a disservice to his pretense towards objectivity.”

100. Assessment: Can’t be checkedSaid 2024-12-11

     “Our best defense against the possible dangers of superintelligence must begin with a realistic appraisal of where we are and how much time we have.”