“The one "big thing" that they've been able to do is to use "reasoning" to make the Large Language Models "think" (they do not have consciousness, they are not "thinking," this just means using more tokens to answer a particular question and having multiple models check the work), which mostly results in them being a bit more accurate when generating an answer, but at the expense of speed and cost.”
Our reading, the sentence that was checked:
The one big thing AI model companies have been able to do is 'reasoning', which means using more tokens and multiple models checking the work rather than actual thinking, and mostly results in models being a bit more accurate at the expense of speed and cost.
Assessment: Mixed Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, The Generative AI Con, paragraph 48.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-longcon-6acf31-a2
We recorded 30 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-02-17. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- mostly (qualifier, stated by the speaker)
From the speaker
- The one "big thing"
- using more tokens
- having multiple models check the work
- mostly results in them being a bit more accurate
- at the expense of speed and cost
- they do not have consciousness
Our interpretation
- Assess the empirical description of reasoning methods and their accuracy, latency and cost tradeoffs in models documented by the post date.
- The context describes developments over several months and possibly years, without a precise starting date. The target period therefore has an open start and ends on the post date.
- The ordinary reading of a bit more accurate is a modest improvement, not merely any positive improvement. I do not invent a numerical cutoff.
- Mostly is a generalization about relevant tasks, not a conditional forecast.
- Multiple models checking the work is read as a description of answer generation, rather than the use of other models during training or evaluation.
- The statements about consciousness and the only big advance lack an agreed test here. They are not treated as established by benchmark evidence.
Counts as supported if: Contemporaneous documentation and representative comparisons establish extra inference tokens and multiple-model checking, predominantly modest accuracy gains, and increased response time and cost.
Counts as contradicted if: Contemporaneous documentation establishes materially different reasoning mechanisms, or suitable comparisons show substantial accuracy gains or an absence of the asserted tradeoff. Evidence can contradict individual components without settling the word mostly.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Mixed
Part of the claim holds and part does not, and the record says which part is which.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Longer generated reasoning is documented in the January 22, 2025 DeepSeek paper, and OpenAI's September 12, 2024 o1-mini release acknowledges that high-capacity models such as o1 can be expensive and slow [e3; e1]. Those parts have support. The multiple-model explanation does not hold as a general definition: DeepSeek describes self-verification within one model's generated reasoning [e3]. The modest-gain description also fails on some documented tasks: OpenAI's September 12, 2024 report gives average single-sample scores on the 2024 AIME exams of 74% for o1 and 12% for GPT-4o [e2]. These selected reasoning benchmarks do not establish what happens across most everyday requests [e2].
Limitations
- The performance results are developer-reported and concentrated on reasoning tasks.
- No representative task distribution settles 'mostly,' and no numerical threshold defines 'a bit.'
- The packet lacks comprehensive matched comparisons of accuracy, latency and customer cost.
- The evidence does not settle consciousness or whether reasoning was the sole major advance.
Evidence (4 items)
Supports the claim: OpenAI,
expensive and slow
Why it matters here: The developer acknowledges cost and speed limitations in the o1 discussion. This supports that part of the claim, but does not establish a universal tradeoff or predominantly small accuracy gains.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Goes against the claim: OpenAI,
these reasoning-heavy tasks
Why it matters here: Appendix A reports AIME pass@1 scores of 44.6 for o1-preview and 9.3 for GPT-4o, showing substantial gains on a demanding reasoning task. These selected evaluations challenge the modest-gain characterization, but cannot determine gains across all ordinary usage.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Goes against the claim: DeepSeek-AI / arXiv,
self-verification, reflection
Why it matters here: The paper describes learned self-verification within a model's generated reasoning. That contradicts treating multiple models checking one another as the defining mechanism, while supporting the use of longer token sequences.
- Publisher
- DeepSeek-AI / arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: OpenAI,
faster performance
Why it matters here: The release documents efficiency improvements within the reasoning-model family. Because the comparator is another reasoning model, this is not a direct test of reasoning versus non-reasoning inference.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, The Generative AI Con
The one "big thing" that they've been able to do is to use "reasoning" to make the Large Language Models "think" (they do not have consciousness, they are not "thinking," this just means using more tokens to answer a particular question and having multiple models check the work), which mostly results in them being a bit more accurate when generating an answer, but at the expense of speed and cost.
- Where
- paragraph 48
- Published
- 2025-02-17
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-longcon-6acf31-a2: Current. Outcome: Mixed. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.