Claims checked against evidence, page 12 of 29
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims checked against evidence (2859)
Each of these claims was checked against dated evidence.
Page 12 of 29: claims 1101 to 1200 of 2859, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Even with thinking mode activated, the output was filled with errors that could easily have been avoided, with a 5-second Wikipedia search that any sensible human could have performed.”
Our reading: GPT-5 in thinking mode produced a list of Canadian Prime Ministers full of easily avoidable errors.
No horizon Evidence as of
Limitation: The retrieved plain-text version of the post does not supply independently verifiable image provenance.
Assessment: ContradictedSaid
Gary Marcus Observation
“They do not sanity check their own work.”
Our reading: LLMs do not sanity check their own work.
No horizon Evidence as of
Limitation: SCoRe used specialized training; its results do not establish that every model checks every answer [e1].
Assessment: MixedSaid
Gary Marcus Observation
“Cherry-picked studies, benchmark-gaming, and now even vibe-coded graphs, with zero transparency about how systems work or how they have been trained; public science is in the rear view mirror.”
Our reading: AI has been marked by cherry-picked studies, benchmark-gaming and now even vibe-coded graphs, with zero transparency about how systems work or how they have been trained, and public science is in the rear view mirror.
No horizon Evidence as of
Limitation: The judgment depends on reading zero transparency as no substantive disclosure, rather than insufficient disclosure for reproduction.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Allegedly godlike models that turn out to be incremental advances.”
Our reading: Models hyped as godlike turn out to be incremental advances.
No horizon Evidence as of
Limitation: A smaller aggregate benchmark gain can coexist with substantial gains in particular uses.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: SupportedSaid
Gary Marcus Observation
“That’s exactly what it means to hit a wall, and exactly the particular set of obstacles I described in my most notorious (and prescient) paper, in 2022. Real progress on some dimensions, but stuck in place on others.”
Our reading: LLM development has hit a wall: real progress on some dimensions but stuck on others.
No horizon Evidence as of
Limitation: The outcome depends on reading stuck as persistence of qualitative failures, as the surrounding passage indicates.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Observation
“The real news is a breaking study from Arizona State University that fully vindicates what I have told you for nearly 30 years—and more recently what Apple told you—about the core weakness of LLMs: their inability to generalize broadly.”
Our reading: LLMs' core weakness is their inability to generalize broadly, as confirmed by an ASU study.
No horizon Evidence as of
Limitation: This assesses the August 5 version available when Marcus wrote.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“Because expectations were through the roof, a huge number of people viewed GPT 5 as a major letdown. By the end of the night, OpenAI’s street cred had dramatically fallen.”
Our reading: Sky-high expectations caused many to see GPT-5 as a major letdown, and OpenAI's credibility fell dramatically on launch night.
No horizon Evidence as of
Limitation: The market movement is documented through news reporting rather than verified original records.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Pure scaling simply isn’t the path to AGI.”
Our reading: Pure scaling simply isn't the path to AGI and will not get us there.
No deadline stated Evidence as of
Limitation: Benchmark achievement is narrower than general intelligence.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Driverless cars that still are only available in couple percent of the world’s cities.”
Our reading: Driverless cars are available in only a couple percent of the world's cities.
No horizon Evidence as of
Limitation: This does not establish that worldwide availability exceeded his estimate.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: SupportedSaid
Gary Marcus Observation
“Many of their best people have left.”
Our reading: Many of OpenAI's best people have left the company.
No horizon Evidence as of
Limitation: Documented prominence does not establish an objective ranking of OpenAI's best employees.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Observation
“What the mainstream media mostly hasn’t told you yet is that a few days later, hardly anybody is buying Altman’s story.”
Our reading: A few days after launch, hardly anybody believes Altman's claims that GPT-5 is like a PhD-level expert.
No horizon Evidence as of
Limitation: The evidence is news reports of individual reactions, not representative data.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Promises for world-changing science that rarely materialize.”
Our reading: Promises that AI will produce world-changing science rarely materialize.
No horizon Evidence as of
Limitation: The searches were not an exhaustive census of AI science promises.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: SupportedSaid
Gary Marcus Observation
“People had grown to expect miracles, but GPT-5 is just the latest incremental advance.”
Our reading: GPT-5 is merely an incremental advance, not a radical leap.
No horizon Evidence as of
Limitation: The result depends on reading the comparison as against the immediately preceding models.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Even some of the tech bros are waking up to the reality that “AGI in 2027” was marketing, not reality.”
Our reading: Even some tech bros are waking up to the reality that 'AGI in 2027' was marketing, not reality.
No horizon Evidence as of
Limitation: The observation concerns public attitudes by August 2025, not whether AGI ultimately arrives in 2027.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Observation
“The challenge of parts and wholes in generative images that Ernest Davis and I discussed here in December fared no better.”
Our reading: GPT-5 shows no improvement on the parts-and-wholes problem in generated images.
No horizon Evidence as of
Limitation: The only systematic test predates GPT-5 and used ChatGPT-4o.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Dozens of shots on goal (Grok, Claude, Gemini) etc have invariably faced the same problems. Distribution shift has never been solved.”
Our reading: Dozens of shots on goal, such as Grok, Claude and Gemini, have invariably faced the same problems, and distribution shift has never been solved.
No horizon Evidence as of
Limitation: Support for recurring weaknesses is stronger than support for the universal wording.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Elon is moving faster. Anthropic and Google and many others are nipping at their heels. Their relationship with Microsoft has frayed.”
Our reading: Elon Musk is moving faster than OpenAI, Anthropic and Google and many others are nipping at OpenAI's heels, and OpenAI's relationship with Microsoft has frayed.
No horizon Evidence as of
Limitation: Release cadence, benchmark rank and rate of capability improvement are different measures.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: SupportedSaid
Gary Marcus Observation
“And, indeed, much as I anticipated here two weeks ago, the problems I have been pointing out over the last quarter century still lingered.”
Our reading: GPT-5 still has the longstanding problems Marcus has identified, such as chess rule-following and world models.
No horizon Evidence as of
Limitation: The main evidence is a single demonstration, not a benchmark.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: ContradictedSaid
Gary Marcus Observation
“The ASU team showed that exactly the same thing was true even in the latest, greatest models. Throw in every gadget invented since 1998, and the Achilles’ Heel I identified then still remains.”
Our reading: The ASU study showed that even the latest, greatest models cannot reliably extend universals outside their training space of examples, so the Achilles' heel Marcus identified in 1998 still remains despite every gadget invented since.
No horizon Evidence as of
Limitation: This does not establish that frontier models generalize reliably.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: UnresolvedSaid
Gary Marcus Observation
“It’s not like other systems are faring much better.”
Our reading: Other generative AI systems such as Grok are not faring much better than GPT-5.
No horizon Evidence as of
Limitation: The packet contains no verified primary comparison from the August 7–9 launch window.
Source: GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
Assessment: SupportedSaid
Gary Marcus Observation
“What was up with multiple graphs (at least two) in which the bars didn’t match the numbers?”
Our reading: In the GPT-5 announcement, OpenAI showed at least two graphs whose bars did not match the numbers.
No horizon Evidence as of
Limitation: The assessment relies on contemporaneous visual analysis and subsequent reporting because the primary video was not readable.
Source: GPT-5 hot take
Assessment: UnresolvedSaid
Gary Marcus Observation
“Pricing is good, but profits may continue to be elusive; still no clear technical moat.”
Our reading: OpenAI still has no clear technical moat with GPT-5.
No horizon Evidence as of
Limitation: Competitor disclosures are self-reported.
Source: GPT-5 hot take
Assessment: SupportedSaid
Gary Marcus Observation
“GPT-5 is obviously not AGI.”
Our reading: GPT-5 is not AGI.
No horizon Evidence as of
Limitation: The outcome depends on the stated interpretation of AGI.
Source: GPT-5 hot take
Assessment: SupportedSaid
Gary Marcus Observation
“OpenAI conveniently forgot to include this comparison (ARC-AGI-2) in their livestream recital of benchmark progress, which left the livestream looking like marketing rather than science.”
Our reading: OpenAI left the ARC-AGI-2 comparison out of its GPT-5 livestream benchmarks, which made the livestream look like marketing rather than science.
No horizon Evidence as of
Limitation: The decisive source is contemporaneous specialist reporting rather than an independently inspected complete recording.
Source: GPT-5 hot take
Assessment: SupportedSaid
Gary Marcus Observation
“But still part of the pack, not a giant leap forward (e.g. Grok 4 beats it on ARC-AGI-2 results)”
Our reading: GPT-5 is still part of the pack rather than a giant leap forward; for example, Grok 4 beats it on ARC-AGI-2.
No horizon Evidence as of
Limitation: The broader conclusion depends on the stated comparison baseline.
Source: GPT-5 hot take
Assessment: UnresolvedSaid
Ed Zitron Observation
“They don’t make enough money to survive without a continuous flow of venture capital, and they don’t seem to make impressive sums of money even when allowed to burn as much as they’d like.”
Our reading: Generative AI startups do not make enough money to survive without a continuous flow of venture capital, and they do not seem to make impressive sums even when allowed to burn as much as they like.
No horizon Evidence as of
Limitation: Neither the broad reading covering generative AI startups nor the narrow reading covering companies discussed in the post is settled.
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Forecast
“At some point capex has to slow — if not because of the lack of revenues or massive costs associated, but because we live in a world with finite space, and when said capex slow happens, so will purchases of NVIDIA GPUs, which will in turn, as proven by Kedrosky and others, slow America’s economic growth.”
Our reading: AI capex will eventually slow, reducing NVIDIA GPU purchases and slowing US economic growth.
No deadline stated Evidence as of
Limitation: Slower growth and lower absolute spending are different tests.
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Observation
“There are standard ways to handle annualized revenue, and it's clear they're not following them.”
Our reading: OpenAI is not following standard methods for calculating annualized revenue.
No horizon Evidence as of
Limitation: The available methodological descriptions are news reports rather than underlying company records.
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Forecast
“That capex reduction will bring with it a reduction in expenditures on NVIDIA GPUs, which will take a chunk out of the US stock market. Although the stock market isn’t the economy, the two things are inherently linked, and the popping of the AI bubble will have downstream ramifications, just like the dot com bubble did on the wider economy.”
Our reading: Reduced AI capex will cut NVIDIA GPU spending, hit the US stock market, and the AI bubble's popping will damage the wider economy like the dot-com bust.
No deadline stated Evidence as of
Limitation: One company's guidance does not establish aggregate realized AI investment.
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Observation
“Large Language Models do not scale profitably, and their functionality is not significant enough to justify the costs of running them.”
Our reading: LLMs don't scale profitably and their functionality doesn't justify their running costs.
No horizon Evidence as of
Limitation: The primary filing covers one provider.
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Conditional forecast
“Expect to see an acceleration in layoffs and offshoring, in part driven by a need for tech companies to show — for the first time in living memory — fiscal restraint.”
Our reading: When the AI bubble pops and capex falls, layoffs and offshoring will accelerate, driven partly by tech companies needing to show fiscal restraint for the first time in living memory.
No deadline stated Evidence as of
Only if: when the AI bubble pops and capex falls
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Forecast
“If I’m honest, I think the future of LLMs will be client-side on egregiously-expensive personal setups for enthusiasts, and in a handful of niche enterprise roles.”
Our reading: He thinks the future of LLMs will be client-side on egregiously expensive personal setups for enthusiasts and in a handful of niche enterprise roles.
No deadline stated Evidence as of
Limitation: The seat-growth figure covers a year, not just the period after the August 2025 prediction [e1].
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Observation
“The reality is simple: the majority of all AI capex is from big tech, which is a massive systemic weakness in our economy.”
Our reading: Most AI capex comes from big tech, which is a major systemic weakness in the US economy.
No horizon Evidence as of
Limitation: The target quarter is an interpretation of wording that mixes quarterly and annual figures.
Source: AI Is A Money Trap
Assessment: ContradictedSaid
Ed Zitron Observation
“We also have had no exits and no IPOs.”
Our reading: Generative AI startups have had no exits and no IPOs.
No horizon Evidence as of
Limitation: The post already acknowledged Leonardo.ai elsewhere; Base44 supplies an additional acquisition example [e1, e2].
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Forecast
“It already has their own extremely-expensive coding environment, Claude Code, which I estimated loses the company 100% to 10,000% of a subscription per-customer a few weeks ago, and now Anthropic is adding weekly limits on accounts, which will, I believe, create some of the most gnarly churn in SaaS history.”
Our reading: He believes Anthropic's new weekly limits on Claude Code accounts will create some of the most gnarly churn in SaaS history.
No deadline stated Evidence as of
Limitation: The implementation evidence confirms existence by March 2026, not the exact initial rollout date.
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Conditional forecast
“While its capex may not have dipped yet (in part because the chip-heavy nature of generative AI means that capex isn’t exclusively dominated by property), it’s now obvious that if it does there will be direct effects on both the US economy and stock market, as Microsoft is part of what amounts to a stimulus package propping up America’s economic growth.”
Our reading: If Microsoft's capex dips, it will directly hurt the US economy and stock market.
No deadline stated Evidence as of
Only if: if Microsoft's capex dips
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Forecast
“Even if this were going to happen — it will not! — who would they give the money to and for how long?”
Our reading: The US government will not give generative AI companies money forever.
No deadline stated Evidence as of
Limitation: The practical reading of 'forever' is an evaluator interpretation.
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Observation
“Why isn’t there one, outside of the companies creating data to train the AI models, or Nvidia? We’re three years in, and we haven’t had one.”
Our reading: Three years in, there is no profitable generative AI company other than training-data firms and Nvidia.
No horizon Evidence as of
Limitation: These are attributed company claims, not verified net-income accounts.
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Conditional forecast
“We’ll see a drying-up of VC funding. Pension funds will take a hit — which will affect how much people have to spend in retirement. It’ll be grim.”
Our reading: When the AI bubble pops, VC funding will dry up and pension funds will take a hit that affects how much people have to spend in retirement, and it will be grim.
No deadline stated Evidence as of
Only if: when the AI bubble pops
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Forecast
“Cursor’s product is now worse. People are going to cancel their subscriptions. Its annualized revenue will drop, and its ability to raise capital will suffer as a direct result.”
Our reading: Cursor's product is now worse, so people will cancel their subscriptions, its annualized revenue will drop, and as a direct result its ability to raise capital will suffer.
No deadline stated Evidence as of
Limitation: Company and investor disclosures are interested-party evidence.
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Ed Zitron Observation
“The complexity of these arrangements makes it hard to anticipate the full economic fallout if the cycle breaks down, but the systemic risk is building.”
Our reading: Systemic risk from private credit financing of AI infrastructure is building.
No horizon Evidence as of
Limitation: The facility announcement does not sufficiently disclose drawdowns, collateral, lender concentration or ultimate exposures [e1].
Source: AI Is A Money Trap
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“The AI industry magic sauce of “scaling” hasn’t made the enshittification situation better; it has made it worse.”
Our reading: Scaling of AI has worsened, not improved, AI-induced enshittification.
No horizon Evidence as of
Limitation: Benchmark results are contextual evidence for a claim about the public information environment.
Source: Slopocalypse Now
Assessment: UnresolvedSaid
Gary Marcus Observation
“That has, of course, only gotten worse. Much worse, as Science reported today.”
Our reading: Generative AI's degradation of science has gotten much worse since early 2024.
No horizon Evidence as of
Limitation: A narrower claim about rising suspicious references has support.
Source: Slopocalypse Now
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“The problem of course is that some significant fraction of that science will be wrong, because of the inevitable tendency of LLMs to hallucinate.”
Our reading: Because LLMs inevitably hallucinate, a significant fraction of science produced with them will be wrong.
No horizon Evidence as of
Limitation: The largest real-world audit cited here is a preprint.
Source: Slopocalypse Now
Assessment: SupportedSaid
Gary Marcus Observation
“He still prefers “enshittification” to “slop”, but the problem is both serious and ubiquitous no matter what you call it.”
Our reading: The problem of AI slop/enshittification is serious and ubiquitous.
No horizon Evidence as of
Limitation: The result depends on the contract's reading of 'serious' and 'ubiquitous', which are evaluative or hyperbolic words.
Source: Slopocalypse Now
Assessment: SupportedSaid
Gary Marcus Observation
“But science is far from alone. AI slop is everywhere. It’s in journalism, law, music (which Nick Cave once called “a grotesque mockery of what it is to be human”), politics, education, you name it.”
Our reading: AI-generated slop is pervasive across journalism, law, music, politics, education and other domains.
No horizon Evidence as of
Limitation: The result depends on reading 'everywhere' as cross-sector breadth.
Source: Slopocalypse Now
Assessment: SupportedSaid
Gary Marcus Observation
“All the big companies have in fact introduced agents.”
Our reading: All the big AI companies have introduced AI agents as of August 2025.
No horizon Evidence as of
Limitation: The company universe is undefined.
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“In practice, those mistakes and errors have greatly limited the utility agents like ChatGPT agent.”
Our reading: Mistakes and errors have greatly limited the utility of agents like ChatGPT agent.
No horizon Evidence as of
Limitation: The causal evidence consists of small, selected task demonstrations.
Assessment: SupportedSaid
Gary Marcus Observation
“The unrelenting hallucinations problem frequently rears its ugly head:”
Our reading: Hallucinations frequently occur in AI agents.
No horizon Evidence as of
Limitation: No standard rate measure; 'frequently' is judged from repeated documented cases.
Assessment: UnresolvedSaid
Gary Marcus Observation
“That sentiment that has only grown over the last several months.”
Our reading: The sentiment that AI agents' reality doesn't match the hype has grown from March to August 2025.
No horizon Evidence as of
Limitation: Trust and disappointment with hype are different measures.
Assessment: SupportedSaid
Gary Marcus Causal claim
“As a consequence of their superficial understanding of what they’re being asked, current agents are grossly vulnerable to cyberattacks.”
Our reading: Current AI agents are grossly vulnerable to cyberattacks because of their superficial understanding of their tasks.
No horizon Evidence as of
Limitation: No study in the packet tests 'superficial understanding' as a cause directly. The causal link rests on how successful attacks worked and on one expert's explanation.
Assessment: UnresolvedSaid
Gary Marcus Observation
“But, none, to my knowledge are reliable, except maybe in very narrow uses cases.”
Our reading: To his knowledge, none of the big companies' AI agents are reliable, except maybe in very narrow use cases.
No horizon Evidence as of
Limitation: The July reports do not provide representative, repeated trials or systematic rates of consequential errors and human intervention [e2, e3].
Assessment: SupportedSaid
Gary Marcus Causal claim
“In fundamentally unreliable systems like LLMs, that means multiple chances for error. Soon or later, those errors catch up. That can and sometimes does lead to catastrophe.”
Our reading: Because LLMs are fundamentally unreliable, multi-step agentic tasks accumulate errors that sometimes lead to catastrophe.
No horizon Evidence as of
Limitation: The evidence is incident reports, not a measured rate of how often multi-step errors cause serious damage.
Assessment: SupportedSaid
Gary Marcus Forecast
“Guardrails (a la ChatGPT) may be in place, but the guardrails will teeter between being too weak (beaten by “jailbreaks”) and too strong (rejecting some perfectly reasonable requests).”
Our reading: Guardrails like ChatGPT's may be in place for GPT-5, but they will teeter between being too weak (beaten by jailbreaks) and too strong (rejecting some perfectly reasonable requests).
Settled before its deadline Evidence as of
Limitation: The evaluations do not establish temporal oscillation, ordinary-user failure rates, or persistence in one unchanged deployment [e2, e3].
Assessment: UnresolvedSaid
Gary Marcus Forecast
“For that, we will, I predict, need genuinely new architectures that incorporate explicit knowledge and world models at their very core.”
Our reading: Trustworthy AI will require genuinely new architectures with explicit knowledge and world models at their core.
No deadline stated Evidence as of
Limitation: Behavior on a bounded benchmark cannot establish broad trustworthiness.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“GPT-5 will still, like its predecessors, be a bull in a china shop, reckless and hard to control.”
Our reading: GPT-5, like its predecessors, will be reckless and hard to control.
No deadline stated Evidence as of
Limitation: METR's task suite does not measure the prevalence or consequences of unintended behavior across ordinary deployments, and its detection methods may miss reward hacking [e1].
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Without external aids it won’t be able beat Meta’s Cicero in Diplomacy; it won’t be able to drive a car reliably; it won’t be able to reliably guide a robot like Optimus to be anything like as versatile as Rosie the Robot.”
Our reading: Without external aids, GPT-5 will not be able to beat Meta's Cicero at Diplomacy, will not be able to drive a car reliably, and will not be able to reliably guide a robot like Optimus to be anything like as versatile as Rosie the Robot.
No deadline stated Evidence as of
Limitation: This compound claim cannot be supported from evidence about robotics alone.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Its natural language output still won’t be something that one can reliably hook up to downstream programs; it won’t be something, for example, that you can simply and directly hook up to a database or virtual assistant, with predictable results.”
Our reading: GPT-5's natural language output will not be something one can reliably hook up to downstream programs, for example simply and directly to a database or virtual assistant, with predictable results.
No deadline stated Evidence as of
Limitation: Distinguishing natural language from plaintext code materially affects the assessment.
Assessment: UnresolvedSaid
Gary Marcus Conditional forecast
“Trustworthy, general artificial intelligence, aligned with human values, will come, when it does, from systems that are more structured, with more built-in knowledge, and will incorporate at least some degree of explicit tools for reasoning and planning, as well as explicit knowledge, that are lacking in systems like GPT.”
Our reading: When trustworthy general AI aligned with human values comes, it will come from systems that are more structured, have more built-in knowledge, and incorporate at least some degree of explicit tools for reasoning and planning, as well as explicit knowledge, that systems like GPT lack.
No deadline stated Evidence as of
Only if: when it does come
Assessment: UnresolvedSaid
Gary Marcus Forecast
“GPT-5 will not have reliable models of the things that it talks about that are accessible to external programmers in a way that reliably feeds downstream processes.”
Our reading: GPT-5 will not have reliable models of the things it talks about that external programmers can access in a way that reliably feeds downstream processes.
No deadline stated Evidence as of
Limitation: The meaning of an accessible subject model materially affects what evidence would count.
Assessment: UnresolvedSaid
Gary Marcus Forecast
“In retrospectives written in 2043, intellectual historians will conclude that there was an initial overemphasis on large language models, and a gradual but critical shift of the pendulum back to more structured systems with deeper comprehension.”
Our reading: In retrospectives written in 2043, intellectual historians will conclude that there was an initial overemphasis on large language models and a gradual but critical shift of the pendulum back to more structured systems with deeper comprehension.
Not yet due Evidence as of
Limitation: Future assessment will require a transparent selection of relevant historians and retrospectives.
Assessment: UnresolvedSaid
Gary Marcus Observation
“I was right on the money then.”
Our reading: Marcus's December 2022 predictions about GPT-4 were accurate.
No horizon Evidence as of
Limitation: The assessment reads the retrospective as referring to all seven predictions; its intended scope is not explicit.
Assessment: SupportedSaid
Gary Marcus Forecast
“Examples of concealed bias will be discovered within days or months.”
Our reading: Examples of concealed bias in GPT-5 will be discovered within days or months of release.
Settled before its deadline Evidence as of
Limitation: The meaning of concealed determines the verdict.
Assessment: SupportedSaid
Gary Marcus Forecast
““Alignment” between what humans want and what machines do will continue to be a critical, unsolved problem.”
Our reading: Alignment between human wants and machine behavior will remain a critical unsolved problem.
Settled before its deadline Evidence as of
Limitation: This supports persistence, not a prediction that alignment will remain unsolved forever.
Assessment: UnresolvedSaid
Gary Marcus Forecast
““Scaling” alone—building bigger and models until they absorb the entire internet — will prove useful, but only to a point.”
Our reading: Scaling alone will prove useful only up to a point.
No deadline stated Evidence as of
Limitation: Finite experiments cannot establish what all future increases in scale will achieve.
Assessment: SupportedSaid
Gary Marcus Observation
“For years we have been hearing, endlessly, about how GPT-5 was going to land imminently, and those predictions turned out to be wrong so often that a year ago I wrote a post about it, called GPT-5…now arriving Gate 8, Gate 9, Gate 10, not to mention a couple of April Fool’s jokes.”
Our reading: Predictions over several years that GPT-5 would arrive imminently were repeatedly wrong.
No horizon Evidence as of
Limitation: Support depends on including qualified public speculation within 'predictions'.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Neither DeepMind nor OpenAI has given any substantial information about how their systems are designed and trained.”
Our reading: Neither DeepMind nor OpenAI has given substantial information on how their IMO systems are designed and trained.
No horizon Evidence as of
Limitation: Substantial is a judgment threshold, not a specified quantity.
Source: DeepMind and OpenAI achieve IMO Gold. What does it all mean?
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Eric Schmidt’s recent prediction (made before IMO 2025), that there will be AI-based “world-class mathematicians” within a year, is pure fantasy.”
Our reading: There will not be AI-based world-class mathematicians within a year (by about July 2026).
Deadline passed Evidence as of
Limitation: The evidence supplies no agreed standard connecting one exceptional result to world-class research capability.
Source: DeepMind and OpenAI achieve IMO Gold. What does it all mean?
Assessment: UnresolvedSaid
Gary Marcus Forecast
“Tying for 27th place in a highly elite high school mathematics competition will not immediately change that.”
Our reading: Tying for 27th place at the IMO will not immediately change the fact that only one theorem of middling importance (the Robbins conjecture, in 1996) has ever been proved autonomously by an AI system.
No deadline stated Evidence as of
Limitation: “Immediately” has no operational endpoint; open_horizon does not extend it indefinitely.
Source: DeepMind and OpenAI achieve IMO Gold. What does it all mean?
Assessment: ContradictedSaid
Gary Marcus Observation
“In the real world, only one theorem of middling importance (the Robbins conjecture) has ever been proved autonomously by an AI system, and that was back in 1996.”
Our reading: Only one theorem of middling importance (the Robbins conjecture, 1996) has ever been proved autonomously by an AI system.
No horizon Evidence as of
Limitation: Mathematical importance is qualitative; the sources do not supply an exact comparison with Robbins.
Source: DeepMind and OpenAI achieve IMO Gold. What does it all mean?
Assessment: SupportedSaid
Gary Marcus Observation
“They have not made clear plans to release the system generally or make it available for experimentation. They have made it clear that it will not be available for months, even in the best case.”
Our reading: OpenAI has not made clear plans to release OpenAI-IMO generally or make it available for experimentation, and has made it clear that it will not be available for months, even in the best case.
No horizon Evidence as of
Limitation: The statement describes intended release timing, not an irreversible commitment.
Source: DeepMind and OpenAI achieve IMO Gold. What does it all mean?
Assessment: SupportedSaid
Gary Marcus Observation
“In contrast, OpenAI communicated very little with the IMO prior to announcing their success.”
Our reading: OpenAI communicated very little with the IMO before announcing its gold-level result.
No horizon Evidence as of
Limitation: The conclusion relies on participant accounts rather than a complete correspondence record.
Source: DeepMind and OpenAI achieve IMO Gold. What does it all mean?
Assessment: ContradictedSaid
Ed Zitron Forecast
“When? Say OpenAI and Broadcom actually build their ASIC in 2026 (they won't) — how many of them will they build?”
Our reading: OpenAI and Broadcom will not build their custom AI ASIC in 2026.
Settled before its deadline Evidence as of
Limitation: The decisive evidence is a company disclosure, not an independently reproduced hardware test.
Assessment: UnresolvedSaid
Ed Zitron Observation
“We’re in a god damn bubble, it is so obvious we’re in a bubble, it’s been so obvious we’re in a bubble, a bubble that seems strong but is actually very weak, with a central point of failure.”
Our reading: AI is obviously in a bubble that looks strong but is very weak, with a central point of failure.
No horizon Evidence as of
Limitation: The cited measurements precede the target market dates and are context only.
Assessment: SupportedSaid
Ed Zitron Forecast
“At some point the momentum behind NVIDIA slows.”
Our reading: NVIDIA's momentum will slow at some point.
Settled before its deadline Evidence as of
Limitation: The result depends on interpreting momentum to include a sustained share-price plateau.
Assessment: UnresolvedSaid
Ed Zitron Observation
“And worse still, it has no path to profitability, and it’s not even anything new!”
Our reading: Perplexity has no path to profitability, and it is not even anything new, being a search engine.
No horizon Evidence as of
Limitation: Financial context comes from a secondary summary of reporting that could not be read.
Assessment: ContradictedSaid
Ed Zitron Observation
“Generative AI does not do the things that it's being sold as doing, and the things it can actually do aren't the kind of things that create business returns, automate labor, or really do much more than one extension of a cloud software platform.”
Our reading: Generative AI does not do what it is marketed as doing, and what it can do neither creates business returns nor automates labor.
No horizon Evidence as of
Limitation: The decisive deployment figure is company-reported through a news outlet, not independently measured.
Assessment: UnresolvedSaid
Ed Zitron Observation
“Someone's gonna get mad at me for saying this, but I believe that Meta is simply burning cash on generative AI.”
Our reading: Meta is simply burning cash on generative AI.
No horizon Evidence as of
Limitation: The reporting period and cash-flow meaning are evaluator interpretations.
Assessment: UnresolvedSaid
Ed Zitron Observation
“Generative AI products do not provide significant revenue growth, its products are not useful in the way that unlocks significant business value, and the products that have some adoption run at such a grotesque loss.”
Our reading: Generative AI products deliver no significant revenue growth or business value, and those with some adoption run at large losses.
No horizon Evidence as of
Limitation: The strongest customer benefit evidence is company-reported.
Assessment: UnresolvedSaid
Ed Zitron Observation
“OpenAI and Anthropic both lose billions of dollars a year after revenue, and their stories do not mirror any other startup in history, not Uber, not Amazon Web Services, nothing.”
Our reading: OpenAI and Anthropic each lose billions of dollars a year after revenue, and their stories do not mirror any other startup in history, including Uber and AWS.
No horizon Evidence as of
Limitation: OpenAI's detailed accounting adjustments are unavailable in the cited report.
Assessment: UnresolvedSaid
Ed Zitron Observation
“I realize it's likely a little boring hearing about software as a service, but this is the only place where generative AI can really make money.”
Our reading: Software as a service is the only place generative AI can really make money.
No horizon Evidence as of
Limitation: The revenue-versus-profit interpretation materially affects the assessment.
Assessment: UnresolvedSaid
Ed Zitron Forecast
“Sam Altman, Dario Amodei, Satya Nadella, Sundar Pichai, Tim Cook, Elon Musk, Mark Zuckerberg and Andy Jassy have overseen a needless, wasteful and destructive economic force that will harm our markets (and by a larger extension our economy) and the tech industry writ large, and when this is over, they must be held accountable.”
Our reading: The AI boom led by these tech CEOs will harm markets, the wider economy and the tech industry.
No deadline stated Evidence as of
Limitation: The market evidence establishes association, not a complete causal estimate.
Assessment: UnresolvedSaid
Ed Zitron Conditional forecast
“OpenAI is Microsoft's largest Azure client — an insanely risky proposition on multiple levels, not simply in the fact that it’s serving the revenue at-cost but that Microsoft executives believed OpenAI would fail in the long term when they invested in 2023 — and Microsoft is NVIDIA's largest client for GPUs, meaning that any changes to Microsoft's future interest in OpenAI, such as reducing its data center expansion, would eventually hit NVIDIA's revenue.”
Our reading: Because Microsoft is NVIDIA's largest GPU client, any changes to Microsoft's future interest in OpenAI, such as reducing its data center expansion, would eventually hit NVIDIA's revenue.
No deadline stated Evidence as of
Only if: if Microsoft's future interest in OpenAI changes, e.g. reducing data center expansion
Assessment: ContradictedSaid
Ed Zitron Observation
“As I wrote earlier in the year, there is really no significant adoption of generative AI services or products.”
Our reading: There is no significant adoption of generative AI services or products.
No horizon Evidence as of
Limitation: The decisive count is company-reported through CNBC, not independently audited.
Assessment: UnresolvedSaid
Ed Zitron Forecast
“There are limited amounts of capital, but also limited amounts of data centers to actually put GPUs, and on top of that, at some point growth will slow at one of the Magnificent 7, at which point costs will have to come down from things that lose them tons of money, such as generative AI.”
Our reading: At some point growth will slow at one of the Magnificent 7, forcing it to cut money-losing costs such as generative AI.
No deadline stated Evidence as of
Limitation: Revenue growth is an evaluator-selected interpretation of growth.
Assessment: UnresolvedSaid
Ed Zitron Observation
“While it theoretically might be, all evidence points to larger models costing more money, especially reasoning-heavy ones like Claude Opus 4.”
Our reading: Although the cost of inference theoretically might be falling, all evidence points to larger models, especially reasoning-heavy ones like Claude Opus 4, costing more money.
No horizon Evidence as of
Limitation: The available studies concern other models and earlier measurements.
Assessment: UnresolvedSaid
Ed Zitron Causal claim
“To be clear, I am not saying that any of the Magnificent 7 are going to die — just that five companies' spend on NVIDIA GPUs largely dictate how stable the US stock market will be.”
Our reading: Five companies' spending on NVIDIA GPUs largely determines how stable the US stock market is, though none of the Magnificent 7 is necessarily going to die.
No horizon Evidence as of
Limitation: The cited assessments predate the target date and are context only.
Assessment: UnresolvedSaid
Gary Marcus Observation
“None of this is inevitable in LLMs. Most other major LLMs don’t share these specific antisocial tendencies, at least not without much more focused efforts at jailbreaking.”
Our reading: Grok's antisemitism, fondness for Hitler and sexual-violence outputs are not inevitable in LLMs, and most other major LLMs don't share these specific antisocial tendencies, at least not without much more focused efforts at jailbreaking.
No horizon Evidence as of
Limitation: CNN does not provide individual trial dates, a complete account of model versions and configurations, or a consistent measure of attack effort [e1].
Assessment: UnresolvedSaid
Gary Marcus Observation
“The system is so bound to Musk it even asked his opinion on pizza.”
Our reading: Grok is so bound to Musk that it sometimes searches for Musk's opinion before answering, and in one case even asked his opinion on pizza.
No horizon Evidence as of
Limitation: The available demonstration establishes occurrence, not how often the behavior happened or its underlying cause.
Assessment: UnresolvedSaid
Gary Marcus Observation
“Grok itself is a long way from AGI, and hardly smart enough to map out an effective world domination.”
Our reading: Grok is a long way from AGI and hardly smart enough to map out an effective world domination.
No horizon Evidence as of
Limitation: The ARC result concerns one benchmark and is accessed through a reproduction of the operator's announcement [e1].
Assessment: UnresolvedSaid
Gary Marcus Observation
“From this I infer that xAI’s main methodology for handling alignment is trial-and-error.”
Our reading: From xAI's account of the incident, Marcus infers that xAI's main methodology for handling alignment is trial-and-error.
No horizon Evidence as of
Limitation: Public disclosures may omit important internal practices.
Assessment: UnresolvedSaid
Gary Marcus Observation
“My p(doom) is still lower than most industry people’s.”
Our reading: Marcus's p(doom) is lower than that of most AI industry people.
No horizon Evidence as of
Limitation: A researcher sample is not automatically representative of the AI industry.
Assessment: UnresolvedSaid
Gary Marcus Observation
“And, currently, we have almost no regulation around AI, for that matter the billions of robots he aspires to build.”
Our reading: There is currently almost no regulation around AI, or around the billions of robots Musk aspires to build.
No horizon Evidence as of
Limitation: Both dated sources precede July 15 and do not provide a complete legal snapshot for that day [e1, e2].
Assessment: UnresolvedSaid
Gary Marcus Observation
“In contrast, o3, when it draws on symbolic code (which it sometimes displays explicitly) can build crossword grids like this far faster than I could (short of my writing my own code).”
Our reading: When o3 draws on symbolic code, it can build crossword grids like this one far faster than Marcus could, short of writing his own code.
No horizon Evidence as of
Limitation: The informal experiment may have private records that are not publicly available.
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: UnresolvedSaid
Gary Marcus Observation
“And it is about why, in 2025, neurosymbolic AI has emerged as the team to beat.”
Our reading: In 2025 neurosymbolic AI has emerged as the leading approach in AI.
No horizon Evidence as of
Limitation: The claim supplies neither a benchmark set nor a definition of overall leadership.
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: MixedSaid
Gary Marcus Observation
“Neural networks are good at learning but weak at generalization; symbolic systems are good at generalization, but not at learning.”
Our reading: Neural networks learn well but generalize poorly, while symbolic systems generalize well but learn poorly.
No horizon Evidence as of
Limitation: The studies do not provide a common comparison across all neural and symbolic methods [e1–e3].
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: SupportedSaid
Gary Marcus Observation
“Systems that draw on code interpreter, and hence by definition are neurosymbolic, often outperform those that lack symbolic tools, at least in a bunch of tasks. This is confirmation of the value of neurosymbolic AI.”
Our reading: Systems using code interpreter often outperform those without symbolic tools on many tasks, confirming the value of neurosymbolic AI.
No horizon Evidence as of
Limitation: The evidence comes from one preprint and selected benchmarks; it does not establish how frequently tools help across real-world tasks [e1].
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: SupportedSaid
Gary Marcus Observation
“Three years later, the pure scaling of pretraining data (and compute, which is to say more and more GPUs) simply hasn’t worked. The specific obstacles that I dwelled on, with respect to hallucinations, misalignment and reasoning errors, have not been overcome.”
Our reading: Pure scaling of pretraining data and compute has not worked; hallucinations, misalignment and reasoning errors remain.
No horizon Evidence as of
Limitation: The result depends on reading 'hasn’t worked' as failure to overcome the named problems.
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: UnresolvedSaid
Gary Marcus Causal claim
“Very little of what has been tried has been discussed explicitly, and because the companies are often quite closed about what they are doing, the public science of neurosymbolic AI is greatly impoverished.”
Our reading: Company secrecy has greatly impoverished public science on neurosymbolic AI.
No horizon Evidence as of
Limitation: The transparency audit does not isolate neurosymbolic methods or their research impact [e1].
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI
Assessment: SupportedSaid
Gary Marcus Observation
“As we will see in a moment, people are finally pursuing a tool-driven, neurosymbolic vision like this — and getting better results than they were able to get with pure neural networks.”
Our reading: Tool-driven neurosymbolic approaches are now being pursued and yield better results than pure neural networks.
No horizon Evidence as of
Limitation: The decisive studies were preprints reporting the authors' own experiments.
Source: How o3 and Grok 4 Accidentally Vindicated Neurosymbolic AI