Claims that can’t be checked, page 36 of 37
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims that can’t be checked (3677)
Public evidence cannot settle these claims, so they are recorded and attributed, not scored. Most were screened out: both AI providers read the claim on its own and found it a value judgment, rhetoric, hedged, too vague to test, or about something no public evidence could show. The rest were found to be recommendations or too open to test in a full review.
Page 36 of 37: claims 3501 to 3600 of 3677, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“I take that as a hopeful sign that the scaling-über-alas narrative is losing force, and that more and more people are open to new things.”
Our reading: Marcus takes Meta's Cicero as a hopeful sign that the scaling-über-alles narrative is losing force and that more and more people are open to new things.
No horizon
Qualifier: I take that as a hopeful sign
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Bias is likely to be huge issue, too.”
Our reading: Bias is likely to be a huge issue with large language models like ChatGPT.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Even if we need to invent a new chip type as a prerequisite, that could be a thirty year process rather a one hundred year process.”
Our reading: Even if a new chip type is required for AGI, that could take about thirty years rather than a hundred.
No horizon
Only if: if a new chip type is needed as a prerequisite
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Of course nothing is guaranteed; we might not get there because we blow ourselves up, or because we waste so much time on large language models that other important challenges—like semantics, causality, commonsense reasoning, abduction and so forth—never get the attention they deserve.”
Our reading: Nothing is guaranteed: we might not get to AGI because we blow ourselves up, or because we waste so much time on large language models that other important challenges such as semantics, causality, commonsense reasoning and abduction never get the attention they deserve.
No horizon
Only if: we blow ourselves up
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“I think that better AI, in which we could directly instill values, might actually make us safer than the world in which we inhabit now, in which so much of what happens is down to the vagaries of what happens to be in this or that training set”
Our reading: Marcus thinks better AI, in which we could directly instill values, might actually make us safer than the current world, in which so much depends on the vagaries of what happens to be in this or that training set.
No horizon
Only if: better AI in which we could directly instill values
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“it terrifies me that so much of current AI is dependent on random details of training corpora that are not available to scientists for inquiry. This seems like a really bad idea.)”
Our reading: Current AI should not depend on random details of training corpora that scientists cannot inspect; this is a bad idea.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“What I think we are missing are (a) ideas and (b) large scale databases of machine-interpretable knowledge.”
Our reading: What AGI development is missing is new ideas and large-scale databases of machine-interpretable knowledge.
No horizon Evidence as of
Limitation: The narrower factual question of whether large knowledge databases existed is checkable.
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“But certainly the hunger is there, the funding is there, and my guess is that patience for LLMs (which I see as a detour) is not infinite.”
Our reading: LLMs are a detour, and the field's patience for them will likely run out.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Real AI wouldn’t need sweatshops.”
Our reading: Genuine AI would not require sweatshop labor.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Sometimes the slower road is the better road.”
Our reading: In choosing what kind of AI to build, sometimes the slower road is the better road.
No horizon
Qualifier: sometimes
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Real AI wouldn’t need vast amounts of data.”
Our reading: Genuine AI would not require vast amounts of training data.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“AI Sweatshops are the consequence of trying to substitute shallow data for deep understanding.”
Our reading: AI sweatshops result from substituting shallow data for deep understanding in AI design.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Friends don’t let friends co-author with ChatGPT.”
Our reading: Scientists should not co-author papers with ChatGPT.
No horizon
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“You wouldn’t make your spell-checker or your spreadsheet a co-author.”
Our reading: ChatGPT should not be made a co-author, just as you wouldn't make your spell-checker or spreadsheet a co-author.
No horizon
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“If you treat ChatGPT as a sentient being, when it clearly is not, you are misleading the public into thinking ChatGPT is sentient; rather than communicating good science, you are hyping bullshit. Just stop.”
Our reading: ChatGPT is clearly not sentient, and scientists should stop treating it as such because doing so misleads the public.
No horizon
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Given ChatGPT’s current understanding level of scientific understanding, assigning authorship to it is disrespectful to the scientific process itself.”
Our reading: Given its current scientific understanding, assigning authorship to ChatGPT disrespects the scientific process.
No horizon
Qualifier: given current level of scientific understanding
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“If you declare it as a co-author, it says you are more interested in being on trend, than in being on target.”
Our reading: Declaring ChatGPT a co-author says that you are more interested in being on trend than in being on target.
No horizon
Qualifier: if you declare it as a co-author
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“I sincerely hope this trend won’t last.”
Our reading: The trend of listing ChatGPT as a co-author should not continue.
No horizon
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“ChatGPT is a tool, but not a person. It’s at best, more like a spell-checker, or a grammar-checker, or a stats package, than it is like a scientist.”
Our reading: ChatGPT is a tool, closer to a spell-checker or stats package than to a scientist.
No horizon
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“The worst thing about ChatGPT’s close-but-no-cigar answer is not that it’s wrong. It’s that it seems so convincing.”
Our reading: The worst thing about ChatGPT's wrong answer in the cited tweet is not that it is wrong but that it seems so convincing.
No horizon
Scope: ChatGPT's close-but-no-cigar answer in the cited tweet
Source: Scientists, please don’t let your chatbots grow up to be co-authors
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“It doesn’t quite add up.”
Our reading: OpenAI preparing to take a lot of money off the table doesn't quite add up if Altman genuinely thinks OpenAI is close to AGI.
No horizon
Scope: if Altman genuinely thinks OpenAI is close to AGI
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“As I explained to Ezra Klein, and have discussed before in this Substack I personally think the latter, less rosy picture, is closer to the mark.”
Our reading: The less rosy picture—that ChatGPT is very limited and not close to AGI—is closer to the truth than the view that OpenAI is near AGI.
No horizon Evidence as of
Limitation: This finding does not reject the factual content about contemporary ChatGPT's limitations.
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Maybe Altman is getting out while the getting is good.”
Our reading: Maybe Altman is getting out of OpenAI while the getting is good.
No horizon
Qualifier: maybe
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“My best guess? Altman doesn’t really know how to make OpenAI into the juggernaut that everybody else seems to think he’s got.”
Our reading: Marcus's best guess is that Altman does not know how to turn OpenAI into the juggernaut others believe it is.
No horizon
Qualifier: best guess
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“In our view, it is past time to have something like this, given the inherent unpredictability and wide range of the systems.”
Our reading: In the authors' view, it is past time to have a community-facing corpus of ChatGPT and other LLM errors, given the systems' inherent unpredictability and wide range.
No horizon
Qualifier: in our view
Source: Large Language Models like ChatGPT say The Darnedest Things
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“George, like Chomsky and myself, called for a greater emphasis on the understanding of human intelligence.”
Our reading: AI research should place greater emphasis on understanding human intelligence.
No horizon
Source: An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“I, Gary Marcus, worried about whether contemporary approaches to AI would ever give solutions to four key aspects of thought that we ought to expect from any intelligent machine: reasoning, abstraction, compositionality, and factuality.”
Our reading: Contemporary approaches to AI may never solve reasoning, abstraction, compositionality, and factuality, which any intelligent machine should have.
No horizon
Source: An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Kai-Fu Lee, often far more optimistic than I am, worried about whether recent advances in AI-generated content were a net positive for society, or whether they might lead to an avalanche of misinformation [something I too am deeply worried about] and target advertising that might be detrimental.”
Our reading: Recent advances in AI-generated content may lead to an avalanche of misinformation.
No horizon
Source: An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“If you want to know what a large cross section of the field’s leaders are thinking, right now, there’s no better place to start, a time capsule of 2022 as we enter into the remaining wild AI years of the 2020s.”
Our reading: The remaining years of the 2020s will be wild for AI.
No horizon
Source: An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“David Ferrucci, CEO of Elemental Cognition, and director of IBM’s successful Watson Jeopardy effort, worried that current systems were “ultimately unsatisfied”, and like me called for hybrid approaches that combine neural networks with reasoning and structured representations.”
Our reading: AI should pursue hybrid approaches combining neural networks with reasoning and structured representations.
No horizon
Source: An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“The trouble is, those guardrails don’t come for free; the system is still shallow; ChatGPT doesn’t really know what it is it is guarding against; it mostly seems instead to be just looking for keywords, which is why it is prone to generating nonsense like this:”
Our reading: ChatGPT's guardrails don't really understand what they guard against and appear to rely mostly on keywords, which makes ChatGPT prone to producing nonsense.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“There’s another problem, too, arguably even bigger concern in the context of search engines: hallucination.”
Our reading: For search engines, hallucination is a further problem for LLM-based systems and arguably an even bigger concern than the shallowness, or discomprehension, he described just before.
No horizon
Scope: in the context of search engines
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“But until those bits all fit together in a way we can trust, I prefer to borrow a memorable phrase from Ariana Grande: Thank U, Next.”
Our reading: Until the pieces of LLM-based search fit together in a way that can be trusted, he personally prefers to pass on it ('Thank U, Next').
No horizon
Qualifier: until those bits all fit together in a way we can trust
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“In the quest for a newfangled search I suspect we are once again in for a rough ride.”
Our reading: Developing LLM-based search will probably be a difficult, drawn-out process, as with driverless cars.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“When it comes to LLM-driven search, caveat emptor!”
Our reading: Users should be cautious when relying on LLM-driven search.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“It is insidious the way that truth and falsity are so thoroughly and authoritatively mixed together. I for one am not ready for our post-truth information overlords.”
Our reading: The way truth and falsity are thoroughly and authoritatively mixed together in LLM-driven search answers like the ones he examined (such as the Neeva bio) is insidious, and he personally is not ready for 'post-truth information overlords'.
No horizon
Scope: LLM-driven search outputs like the examples he examined, such as the Neeva bio
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“I like to call this sort of nonsense discomprehension: mindless answers that show that the systems has no clue what it is that is talking about.”
Our reading: ChatGPT's mindless answers ('discomprehension') show it has no clue what it is talking about.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“I suspect that what we will see is a familiar pattern: immense initial buzz, followed by a more careful scientific inspection, followed by a recognition that many problems remain.”
Our reading: GPT-4 will likely generate immense buzz followed by scientific scrutiny and recognition that many problems remain.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“When AGI (artificial intelligence) comes, large language models like GPT-4 may be seen in hindsight as part of the eventual solution, but only as part of the solution.”
Our reading: When AGI comes, large language models like GPT-4 may be seen in hindsight as part of the eventual solution, but only as part of it.
No horizon
Only if: when AGI comes
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“If all seven predictions prove correct, I hope that the field will finally realize that it is time to move on.”
Our reading: Marcus hopes that, if all seven of his predictions prove correct, the field will finally realize that it is time to move on.
No horizon
Only if: if all seven predictions prove correct
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“And it’s clear that that is exactly what large language models make possible. They are aiming to create a world in which we are unable to know what we can trust; with these new tools, they might succeed.”
Our reading: LLMs make greatly escalated propaganda volume possible, and propagandists might succeed in creating a world where people cannot know what to trust.
No horizon
Source: AI's Jurassic Park moment
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Executives at Meta and OpenAI are as enthusiastic about their tools as the proprietors of Jurassic Park were about theirs.”
Our reading: Executives at Meta and OpenAI are as enthusiastic about their tools as the proprietors of Jurassic Park were about theirs.
No horizon
Source: AI's Jurassic Park moment
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“It is no exaggeration to say that systems like these pose a real and imminent threat to the fabric of society.”
Our reading: Generative AI systems like chatGPT pose a real and imminent threat to the fabric of society.
No horizon
Source: AI's Jurassic Park moment
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Large language models lack mechanisms for verifying truth; we need to find new ways to integrate them with the tools of classical AI, such as databases, webs of knowledge, and reasoning.”
Our reading: We need to integrate LLMs with classical AI tools such as databases, knowledge graphs, and reasoning to fight misinformation.
No horizon
Source: AI's Jurassic Park moment
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Nation-states and other bad actors that deliberately produce propaganda, are highly unlikely to voluntarily put down their new arms. Instead, they are likely to use large language models as a new class of automatic weapons, in their war on truth, attacking social media and crafting fake web sites at a volume we have never seen before.”
Our reading: Propaganda-producing nation-states and bad actors are likely to use LLMs to attack social media and create fake websites at unprecedented volume.
No horizon
Source: AI's Jurassic Park moment
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“In time, though it would not be a popular decision, we may have to begin to treat misinformation as we do libel, making it actionable if it is created with sufficient malice and sufficient volume.”
Our reading: Countries may need to make misinformation legally actionable like libel when created with sufficient malice and volume.
No horizon
Qualifier: if created with sufficient malice and volume
Source: AI's Jurassic Park moment
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“First, every social media company and search engine should support and extend StackOverflow’s ban; automatically-generated content that is misleading, should not be welcome, and the regular posting of it should be grounds for a user’s removal.”
Our reading: Every social media company and search engine should support and extend Stack Overflow's ban, misleading automatically-generated content should not be welcome, and regularly posting it should be grounds for a user's removal.
No horizon
Scope: automatically-generated content that is misleading
Source: AI's Jurassic Park moment
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“User accounts must be more strenuously validated, and new systems like Harvard and Mozilla’s human-ID.org that allow for anonymous, bot-resistant authentication need to become mandatory; they are no longer a luxury we can afford to wait on.”
Our reading: User accounts must be more strongly validated and anonymous bot-resistant authentication should become mandatory.
No horizon
Source: AI's Jurassic Park moment
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“As one of the most central resources that the world’s programmers rely on, the consequences for software quality and developer productivity could be immense.”
Our reading: If Stack Overflow is flooded with worthless ChatGPT-generated code examples and dies, then, because it is one of the most central resources the world's programmers rely on, the consequences for software quality and developer productivity could be immense.
No horizon
Only if: if Stack Overflow is flooded with worthless generated code examples and dies
Source: AI's Jurassic Park moment
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“GPT-3 has no idea how the world works (and on this Bender and I would agree);”
Our reading: GPT-3 does not understand how the world works.
No horizon
Source: How come GPT can seem so brilliant one minute and so breathtakingly dumb the next?
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“But it is mimic that knows not whereof it speaks.”
Our reading: GPT-3 is a mimic that does not understand what it is saying.
No horizon
Source: How come GPT can seem so brilliant one minute and so breathtakingly dumb the next?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“It was just named a Best Tech Article of 2022, and I believe that is still incredibly relevant, even with all the advances of the last 8 months.”
Our reading: Marcus believes his essay 'Deep Learning is Hitting a Wall' is still incredibly relevant, even with all the advances of the last 8 months.
No horizon
Qualifier: I believe
Source: How come GPT can seem so brilliant one minute and so breathtakingly dumb the next?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“The results themselves are, without question, genuinely impressive.”
Our reading: Cicero's Diplomacy results are, without question, genuinely impressive.
No horizon
Qualifier: without question
Source: What does Meta AI’s Diplomacy-winning Cicero Mean for AI?
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“(The contest was a “blitz” version of the game, limited to five minutes a move, which favors computers, since the blitz version is presumably less dialogue- and negotiation-intensive than the full version of the game; in a longer version of the game, humans may still dominate.)”
Our reading: The five-minute-a-move blitz version of Diplomacy favors computers, presumably because it is less dialogue- and negotiation-intensive than the full game, and in a longer version humans may still dominate.
No horizon
Qualifier: presumably
Source: What does Meta AI’s Diplomacy-winning Cicero Mean for AI?
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Our best guess is that if you played Diplomacy with alternative rules, the system would like to have retrained almost from scratch,”
Our reading: If Diplomacy were played with alternative rules, Cicero would probably need to be retrained almost from scratch.
No horizon
Only if: if Diplomacy were played with alternative rules
Source: What does Meta AI’s Diplomacy-winning Cicero Mean for AI?
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Cicero may change that calculus.”
Our reading: Cicero may change the field's view of machine learning as a universal solvent.
No horizon
Source: What does Meta AI’s Diplomacy-winning Cicero Mean for AI?
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Cicero is in many ways a marvel; it has achieved by far the deepest and most extensive integration of language and action in a dynamic world of any AI system built to date.”
Our reading: Cicero is in many ways a marvel and has achieved by far the deepest and most extensive integration of language and action in a dynamic world of any AI system built to date.
No horizon Evidence as of
Limitation: This does not dispute that Cicero represented a substantial technical achievement.
Source: What does Meta AI’s Diplomacy-winning Cicero Mean for AI?
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“as things stand now, the prospects for generalization seem to us to be somewhat limited; the style of work might well be useful in other problems, but it may be that not much of the specifics of the architecture would survive if the system were applied to other problems”
Our reading: Saying he cannot predict this with any great confidence, Marcus judges that, as things stand now, Cicero's prospects for generalization seem somewhat limited: its style of work might well be useful in other problems, but it may be that not much of its specific architecture would survive if the system were applied to other problems.
No horizon
Only if: if the system were applied to other problems
Source: What does Meta AI’s Diplomacy-winning Cicero Mean for AI?
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“This one may well go on for a long time, advancing as technology itself advances.”
Our reading: The argument over whether AI systems are sentient may well go on for a long time, evolving as the technology advances.
No horizon
Qualifier: advancing as technology advances
Source: Sentience and AI: A Dialog between Gary Marcus and Blake Lemoine
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Consciousness is probably like that; we probably can’t really answer the questions we are striving for, at least not yet; we have to satisfy ourselves with trying to understand the arguments clearly.”
Our reading: Questions about consciousness probably cannot yet really be answered, given current science and technology, so we must settle for understanding the arguments clearly.
No horizon
Qualifier: at least not yet
Source: Sentience and AI: A Dialog between Gary Marcus and Blake Lemoine
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“they are not (and may never be) great tech for reliably inferring user intent from what users say.”
Our reading: LLMs may never become great technology for reliably inferring user intent.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“LLMs are inherently unreliable. If Alexa were to make frequent errors, people would stop using it.”
Our reading: Marcus suspects that a contributing reason Alexa does not converse is that LLMs are inherently unreliable, and if Alexa made frequent errors, people would stop using it.
No horizon
Only if: if Alexa were to make frequent errors
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Any one of these scenarios of LLMs gone awry (bad advice, insults, lies etc) could hurt the Amazon brand, open up litigation, etc..”
Our reading: Any one of the scenarios of LLMs going awry (bad advice, insults, lies, etc.) could hurt the Amazon brand and open up litigation.
No horizon
Only if: any one of these scenarios of LLMs gone awry
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Amazon probably *is* using large language models for product search.”
Our reading: Amazon is probably using large language models for product search.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“No matter how much data you can collect, don’t count on deep learning.”
Our reading: Twitter should not count on deep learning to detect misinformation, regardless of how much data it collects.
No horizon
Qualifier: regardless of data quantity
Source: Bravo @ElonMusk for wanting Twitter to become the world’s “most accurate source of information”
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“You are going to need AI. But not (just) the popular stuff.”
Our reading: Twitter will need AI to fight misinformation, but not just popular AI approaches.
No horizon
Source: Bravo @ElonMusk for wanting Twitter to become the world’s “most accurate source of information”
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“You’re going to need something different.”
Our reading: Fighting misinformation on Twitter requires an AI approach different from deep learning/LLMs.
No horizon
Source: Bravo @ElonMusk for wanting Twitter to become the world’s “most accurate source of information”
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“The proof makes too many assumptions to be taken as gospel, but if it is even close to correct, there may soon be real trouble in Scaling City.”
Our reading: If the Merrill, Warstadt and Linzen proof is even close to correct, scaling maximalism may soon be in real trouble.
No horizon
Only if: if the Merrill/Warstadt/Linzen proof is even close to correct
Source: What “game over” for the latest paradigm in AI might look like
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“If you get syntax and semantics but fail on pragmatics or commonsense reasoning, you don’t have AGI you can trust.”
Our reading: A system that gets syntax and semantics right but fails on pragmatics or commonsense reasoning is not AGI you can trust.
No horizon
Scope: system gets syntax and semantics but fails on pragmatics or commonsense reasoning
Source: What “game over” for the latest paradigm in AI might look like
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“All of this points to the same conclusion: if we want to reach AGI we shouldn’t keep putting so many eggs in the scaling-über-alles basket.”
Our reading: If the goal is AGI, the AI field should stop putting so much of its effort into scaling alone.
No horizon
Qualifier: if we want to reach AGI
Source: What “game over” for the latest paradigm in AI might look like
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“There might not be enough data in the world to make scaling maximalism work.”
Our reading: There may not be enough data in the world for scaling maximalism to work.
No horizon
Source: What “game over” for the latest paradigm in AI might look like
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“What hit me hard, as I was reading the paper, is that asymptotic 80% performance on even a single important task like this might spell game over for scaling.”
Our reading: If performance plateaus at about 80% on even one important task like pragmatics, that may mean game over for scaling.
No horizon
Only if: if performance plateaus at about 80% on a single important task
Source: What “game over” for the latest paradigm in AI might look like
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“There might not be enough available compute in the world to make scaling maximalism feasible.”
Our reading: There may not be enough available compute in the world to make scaling maximalism feasible.
No horizon
Source: What “game over” for the latest paradigm in AI might look like
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Some important tasks might simply not scale.”
Our reading: Some important tasks may simply not improve with scale.
No horizon
Source: What “game over” for the latest paradigm in AI might look like
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“Knowing more about exactly what it is beyond size that makes our mind unique would tell us an immense amount, both about ourselves, and about how we might want to build our AI.”
Our reading: Knowing more about exactly what it is, beyond brain size, that makes the human mind unique would tell us an immense amount both about ourselves and about how we might want to build AI.
No horizon
Qualifier: knowing more about exactly what beyond size makes our mind unique
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“Undoubtedly clever intelligences than ours are possible and may yet materialize.”
Our reading: Intelligences cleverer than human intelligence are possible and may eventually materialize.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“MetaAI’s new text-to-movie software Make a Video is straight-up amazing.”
Our reading: Meta AI's new text-to-video software Make a Video is straight-up amazing.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Causal claim
“i disagree w this logic: building biggest model isn’t inherently a contribution to the innovations we need”
Our reading: Having the most data and compute and building the biggest model is not inherently a contribution to the innovations needed for AGI.
No horizon
Source: Sub-Optimal
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“With respect to raw computational power, Tesla might eventually outgun Boston Dynamics, depending on how things go, but I seriously doubt they could outgun Google if Google went all in on humanoid robotics.”
Our reading: Tesla might eventually exceed Boston Dynamics in compute, but very likely could not outgun Google if Google went all in on humanoid robotics.
No horizon
Only if: if Google went all in on humanoid robotics
Source: Sub-Optimal
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“(Though he clearly has not invested nearly enough so far.)”
Our reading: Musk clearly has not, so far, invested nearly enough to win the robotics race.
No horizon
Scope: so far
Source: Sub-Optimal
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“Meta, which has more data and compute than almost anyone, has finally woken up to that. Musk still hasn’t.”
Our reading: Meta has recognized that data and compute alone won't suffice, while Musk has not.
No horizon
Source: Sub-Optimal
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“With enough investment, Tesla might in principle catch up. If Musk really wants to win the robotics race, he has the resources to do so.”
Our reading: With enough investment, Tesla might in principle catch up to Boston Dynamics in humanoid robotics.
No horizon
Only if: with enough investment
Source: Sub-Optimal
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“The most important concern is that robotics is always harder than people think it is.”
Our reading: Robotics is always harder than people think it is.
No horizon
Source: Top 5 reasons Tesla’s Optimus announcement may turn out be a dud
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Roboticists are likely to scoff unless they see real evidence of progress than on Friday.”
Our reading: Roboticists are likely to scoff at Tesla's Friday Optimus showing unless they see real evidence of progress.
No horizon
Only if: unless they see real evidence of progress
Source: Top 5 reasons Tesla’s Optimus announcement may turn out be a dud
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“The fleets of driverless taxis he promised are likely still years or decades away.”
Our reading: The fleets of Tesla driverless taxis Musk promised are likely still years or decades away as of September 2022.
No horizon
Source: Top 5 reasons Tesla’s Optimus announcement may turn out be a dud
Assessment: Can’t be checkedSaid
Gary Marcus Forecast
“None of this is to say that Tesla could not solve all of these problems eventually given enough time and money, but Friday may not be the big demo his fans are hoping for.”
Our reading: Tesla's Optimus demo on Friday, September 30, 2022 may not be the big demo Musk's fans are hoping for.
No horizon
Source: Top 5 reasons Tesla’s Optimus announcement may turn out be a dud
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“In my view the Google demo is pretty problematic (despite being cool), particularly in terms of likely reliability, but no matter.”
Our reading: In Marcus's view, Google's demo hooking a large language model to a robot, though cool, is pretty problematic, particularly in terms of likely reliability.
No horizon
Qualifier: in my view
Source: Top 5 reasons Tesla’s Optimus announcement may turn out be a dud
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“A large part of LeCun’s new manifesto is a well-motivated call for incorporating a “configurable predictive world model” into deep learning.”
Our reading: Deep learning should incorporate configurable predictive world models.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“I won’t accuse LeCun of plagiarism, because I think he probably reached these conclusions honestly, after recognizing the failures of current architectures.”
Our reading: Marcus thinks LeCun probably reached his conclusions honestly, after recognizing the failures of current architectures, and so he will not accuse LeCun of plagiarism.
No horizon
Qualifier: I think he probably
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“It may be boring to hear about the same challenges over and over, but it’s hard to see how we will get to trustworthy general intelligence until these problems are resolved, in ways that are robust, rather than merely statistically contingent.”
Our reading: It is hard to see how we will get to trustworthy general intelligence until the challenges he has repeatedly raised are resolved in ways that are robust rather than merely statistically contingent.
No horizon
Only if: until these problems are resolved in ways that are robust rather than merely statistically contingent
Assessment: Can’t be checkedSaid
Gary Marcus Rhetoric
“Those who wish that AI criticism would go away are welcome to their views, but misrepresenting critics and their views (and obscuring the degree to which even mainstream researchers agree) won’t magically solve the problems AI has yet to solve.”
Our reading: Misrepresenting AI critics and their views, and obscuring the degree to which even mainstream researchers agree, won't magically solve the problems AI has yet to solve.
No horizon
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“In assessing progress towards general intelligence, the critical question should be, how much do systems like Dall-E, Imagen, Midjourney, and Stable Diffusion really understand the world, such that they can reason on and act on that knowledge?”
Our reading: Progress towards general intelligence from image systems like DALL-E, Imagen, Midjourney and Stable Diffusion should be assessed by how much they understand the world so they can reason and act on it.
No horizon
Qualifier: in assessing progress towards general intelligence
Source: Form, function, and the giant gulf between drawing a picture and understanding the world
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“(Dare I make obvious guess? They probably tried and failed, and don’t want to admit it.)”
Our reading: Google probably tested Imagen on Winoground, got poor results, and doesn't want to admit it.
No horizon
Source: Did GoogleAI Just Snooker One of Silicon Valley’s Sharpest Minds?
Assessment: Can’t be checkedSaid
Gary Marcus Observation
“For me, compositionality has always been the touchstone for AI; when I say AI has hit a wall, it’s compositionality, more than anything else, that I have been talking about:”
Our reading: AI has hit a wall, and that wall is mainly compositionality.
No horizon
Source: Did GoogleAI Just Snooker One of Silicon Valley’s Sharpest Minds?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“A will to believe in AI will never replace the need for careful science.”
Our reading: A will to believe in AI will never replace the need for careful science.
No horizon
Source: Did GoogleAI Just Snooker One of Silicon Valley’s Sharpest Minds?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“Having a Silicon Valley celebrity like Alexander weigh in for Google is great PR for Google, but not so great for the public understanding of AI.”
Our reading: Having a Silicon Valley celebrity like Alexander weigh in for Google is great PR for Google but not so great for the public understanding of AI.
No horizon
Source: Did GoogleAI Just Snooker One of Silicon Valley’s Sharpest Minds?
Assessment: Can’t be checkedSaid
Gary Marcus Recommendation or value judgment
“In 2017, a team of Google researchers invented the Transformer, arguably the biggest advance in AI in the last 10 years.”
Our reading: The Transformer, invented by a team of Google researchers in 2017, is arguably the biggest advance in AI in the last 10 years.
No horizon
Qualifier: arguably
Source: Did GoogleAI Just Snooker One of Silicon Valley’s Sharpest Minds?
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Without a much greater degree of interpretability and verifiability, its hard to see how we can get to that level of safety.”
Our reading: Without a much greater degree of interpretability and verifiability, it is hard to see how humanoid home robots can reach the level of safety needed to act reliably and safely on users' requests.
No horizon
Only if: without a much greater degree of interpretability and verifiability
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“Misunderstandings will arise; without really firm checks on these systems, some of those misunderstandings could lead to mayhem.”
Our reading: LLM-driven robots will misunderstand people, and without firm checks some of those misunderstandings could lead to mayhem.
No horizon
Only if: without really firm checks on these systems
Assessment: Can’t be checkedSaid
Gary Marcus Conditional forecast
“It’s not just that large language models can counsel suicide, or sign off on genocide, or that they can be toxic and that they are incredibly (over)sensitive to the details of their training set - it’s that if you put them inside a robot, and they misunderstand you, or fail to fully appreciate the implications of your request, they could cause major damage.”
Our reading: If large language models are put inside robots and misunderstand a request or miss its implications, they could cause major damage.
No horizon
Only if: LLMs put inside a robot