The Sam Altman Playbook
We recorded 18 claims from this post. 8 were checked against evidence. 10 can’t be checked. 5 more are held. Read the post at its source.
Claims from this post (18)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
“At the same talk at Stanford, Altman grandoisely asserted, with no qualification, that “we’re making AGI”, without touching on the possibility that current approaches might not get us there — despite (a) the enormous known problems with reliability, reasoning, planning and hallucinations and (b) the fact that the entire field, despite being massively well-capitalized, has struggled for over year (and in fact for decades) to solve any of these problems.”
What we tested: Current AI approaches have enormous known problems with reliability, reasoning, planning and hallucinations, and the entire field, despite being massively well-capitalized, has struggled for over a year, and in fact for decades, to solve any of these problems.
What happened: OpenAI explicitly acknowledged three named limitations more than a year before Marcus's post. The same section also reports reduced hallucination and improved factuality, so it supports unresolved problems rather than an absence of progress. (OpenAI, )
Assessment: Can’t be checkedSaid
“Whether GenAI proves to be AGI is very much still up for grabs, and on any reasonable account we have a long way to go.”
What we tested: Whether generative AI proves to be AGI is very much still up for grabs, and on any reasonable account there is a long way to go.
Assessment: Can’t be checkedSaid
“At Stanford he said, “I think every year for the next many we have dramatically more capable systems every year”, as if that were a sure thing, when of course it isn’t.”
What we tested: It is not a sure thing that AI systems will become dramatically more capable every year for many years.
Assessment: UnresolvedSaid
“He seems like such a nice guy, yet he implies, unrealistically, that the solution to AGI is within his grasp; he presents no evidence that is so, and rarely considers the many critiques of current approaches that have been raised.”
What we tested: Altman implies, unrealistically, that the solution to AGI is within his grasp, presents no evidence that it is, and rarely considers the many critiques of current approaches that have been raised.
Assessment: UnresolvedSaid
“... but in reality mostly just sells chatbots. Charitable work is at best a small part of the overall operation.”
What we tested: OpenAI mostly sells chatbots, and charitable work is at best a small part of its operation.
Assessment: UnresolvedSaid
“But wait, if he really believes in basic income for all, why is he fleecing artists, writers, and other creators, not paying a nickel (to most) for training materials? A lot of them are losing gigs.”
What we tested: OpenAI does not pay most creators for training materials, and many creators are losing gigs.
Assessment: Can’t be checkedSaid
“Many (except perhaps the humility and soft-spoken tone) are familiar from Elon Musk’s regular overpromising; almost all the AI companies are basically following the same playbook, albeit with less panache.”
What we tested: Almost all AI companies follow the same overpromising, speculative playbook as Altman.
Assessment: UnresolvedSaid
“Usership for ChatGPT rose like a rocket until May 2023 but has more or less stabilized since then without significant further growth.”
What we tested: ChatGPT usership has more or less stabilized without significant growth since May 2023.
Assessment: Can’t be checkedSaid
“If they lost one of those in court, and precedent against training on copyright materials was set, the entire enterprise could be in jeopardy.”
What we tested: If OpenAI lost a copyright lawsuit setting precedent against training on copyrighted materials, its entire enterprise could be in jeopardy.
Assessment: UnresolvedSaid
“So far, there have been no net profits, and as noted above if there are any, for a long time roughly half must be shared with Microsoft, by contractual agreement.”
What we tested: OpenAI has had no net profits so far, and roughly half of any future profits must be shared with Microsoft for a long time.
Assessment: MixedSaid
“In the last year, his company has released no major update to GPT-4, and many competitors have caught up or are close. Competitors are eroding the initial gap in quality and starting to offer similar products, sometimes at significantly lower prices.”
What we tested: In the last year OpenAI has released no major update to GPT-4, many competitors have caught up or are close, and competitors are eroding the initial gap in quality and starting to offer similar products, sometimes at significantly lower prices.
What held: The release documents overlapping multimodal chat and API products and reports strong Opus benchmark performance. Sonnet's published input and output token charges were below GPT-4 Turbo's corresponding charges in e1, although Opus was more expensive and Sonnet was a lower capability tier. (Anthropic, )
What did not: The announcement documents an accessible GPT-4 Turbo preview with 128K context, improved instruction following, JSON mode and image-input support in the API. These constitute a substantial update under the stated reading of major. (OpenAI, )
Assessment: Can’t be checkedSaid
“Altman isn’t out there selling a new product; he is selling hope.”
What we tested: Altman is selling hope rather than a new product.
Assessment: Can’t be checkedSaid
“GoogleDeepMind, Anthropic, Meta or someone else could conceivably beat them there.”
What we tested: Google DeepMind, Anthropic, Meta or someone else could conceivably beat OpenAI to a GPT-5-level model.
Assessment: Can’t be checkedSaid
“Meta will likely release a GPT-4 level model within months – for free.”
What we tested: Meta will likely release a free GPT-4-level model within months of May 2024.
Assessment: Can’t be checkedSaid
“Training GPT-4 was expensive, training GPT-5 maybe an order of magnitude more expensive. GPT-6 yet another order more expensive.”
What we tested: Training GPT-5 may cost an order of magnitude more than GPT-4, and GPT-6 another order of magnitude more.
Assessment: Can’t be checkedSaid
“If OpenAI has a moat (beyond initial customers, which could potentially move), we haven’t seen it.”
What we tested: OpenAI has shown no evident moat beyond its initial customers, who could move.
Assessment: Can’t be checkedSaid
“In seeking to grab vast resources for an approach that is deeply flawed, Altman is diverting resources from alternative approaches from AI that might be more trustworthy, reliable, and interpretable, potentially incurring considerable environment costs, and distracting from better ways of using capital to help humanity more generally.”
What we tested: In seeking vast resources for a deeply flawed approach, Altman is diverting resources from alternative AI approaches that might be more trustworthy, reliable and interpretable, is potentially incurring considerable environmental costs, and is distracting from better ways of using capital to help humanity more generally.
Assessment: SupportedSaid
“People who buy Altman’s narrative will continue to invest.”
What we tested: People who buy Altman's narrative will continue to invest.
What happened: SoftBank explicitly endorses OpenAI's role in achieving advanced general intelligence. This establishes the belief condition under the stated public-endorsement interpretation. (SoftBank Group, )
Held from this post (5)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- Attribution is uncertain
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: OpenAI has not demonstrated a working, reliable agent and does not appear to have a GPT-5-level model as of May 2024.
- The two reviews reached different assessments. Our reading, not assessed: Roughly half of OpenAI's first couple hundred billion dollars in profits, if they materialize at all, will go to Microsoft, and artists, writers, publishers and others are being screwed day after day, their work used without compensation or consent.