𝐩(𝐝𝐨𝐨𝐦)
We recorded 24 claims from this post. 6 were checked against evidence. 18 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (24)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“Personally, despite the name of this essay, I am not all that worried about humans going extinct anytime very soon.”
What we tested: Marcus says he is not all that worried about humans going extinct anytime very soon.
Assessment: Can’t be checkedSaid
“There are in fact already some scenarios that are faintly imaginable now (e.g. bad actors using machines to create novel toxins, in ways that go awry), and over time could be more.”
What we tested: Some faintly imaginable scenarios already exist in which machines could kill everyone, such as bad actors using machines to create novel toxins, and more such scenarios could appear over time.
Assessment: Can’t be checkedSaid
“Perhaps extinction is vanishingly unlikely, but what I will call 𝐩(𝐜𝐚𝐭𝐚𝐬𝐭𝐫𝐨𝐩𝐡𝐞) – the chance that an incident that kills (say) one percent or more of the population, seems a lot higher.”
What we tested: Perhaps extinction is vanishingly unlikely, but the chance of an incident that kills, say, one percent or more of the population seems a lot higher.
Assessment: UnresolvedSaid
“Yet a really large fraction of the AI and AI-adjacent community doesn’t really want to talk about any of this at all.”
What we tested: A large fraction of the AI and AI-adjacent community does not want to discuss AI doom or catastrophe risk at all.
Assessment: Can’t be checkedSaid
“Unfortunately, the fact that we can’t predict the course of technology with any precision even a decade out doesn’t magically immunize us from its effects, nor relieve us of a moral responsbility to try to take precautions.”
What we tested: Not being able to predict technology precisely does not excuse us from the moral responsibility to take precautions against its risks.
Assessment: Can’t be checkedSaid
“LeCun’s conclusion —that humans will always set the agenda—might be true, but he hasn’t given a real argument.”
What we tested: LeCun has given no real argument that humans will always set the agenda over AI, though that conclusion might be true.
Assessment: SupportedSaid
“When I think about how poor his company (Meta) is even at making LLMs that don’t hallucinate, I get worried.”
What we tested: Meta is poor at making LLMs that do not hallucinate.
What happened: The 70B chat model achieved 64.14% generations judged both truthful and informative on TruthfulQA, showing a substantial gap from reliable factual answering. This result is also present in the dated July technical report; unsuccessful answers cannot all be counted as hallucinations. (Meta, )
Assessment: Can’t be checkedSaid
“None of this means of course that I suddenly think that we are close to super-smart machines. (Spoiler alert: we are not).”
What we tested: We are not close to super-smart machines.
Assessment: UnresolvedSaid
“But if and when we get there, will they be dominating? Non-dominating? The honest answer is we haven’t a clue.”
What we tested: Marcus says that, honestly, we haven't a clue whether super-smart machines, if and when we get there, will be dominating or non-dominating.
Assessment: Can’t be checkedSaid
“As Englander well knows, we will never get perfect estimates, but even imperfect estimates might help guide policy decisions as we try to balance attention to short-term and long-term risks in AI.”
What we tested: We will never get perfect estimates of p(doom) or p(catastrophe), but even imperfect estimates might help guide policy decisions as we try to balance attention to short-term and long-term AI risks.
Assessment: Can’t be checkedSaid
“It’s therefore worth getting a rough idea of the number, even if we can’t achieve consensus on the exact number.”
What we tested: It is worth getting a rough estimate of p(doom) even if there is no consensus on the exact number.
Assessment: Can’t be checkedSaid
“But imprecision is not an excuse for inaction.”
What we tested: Imprecise risk estimates do not justify failing to act on AI risk.
Assessment: Can’t be checkedSaid
“Unlike Yudkowsky I don’t think the probabilities are high enough—or imminent enough- to warrant bombing data centers any time soon. (Or probably ever.)”
What we tested: AI doom probabilities are not high or imminent enough to justify bombing data centers any time soon, and probably never will be.
Assessment: UnresolvedSaid
“Some of the specific arguments Yudkowsky and Bostrom have made may well be weak, but there is no solid argument that the number is zero, or even that the risk is infinitesimal.”
What we tested: There is no solid argument that AI doom risk is zero or even infinitesimal, though some arguments by Yudkowsky and Bostrom may be weak.
Assessment: Can’t be checkedSaid
“We waited too long on climate change; to the extent that there may be any real risk, either of outright doom or “mere” catastrophe, let’s not wait too long on AI.”
What we tested: To the extent AI carries any real risk of doom or catastrophe, society should not wait too long to address it, unlike with climate change.
Assessment: Can’t be checkedSaid
“I am, at least for now, much more worried about the overapplication of mediocre AI (eg what happens if we put large language models into running military strategy or into hiring decisions that affect millions of people) than the use of AI that might be so smart it could catch its own errors.”
What we tested: Marcus says that, at least for now, he is much more worried about the overapplication of mediocre AI (e.g., putting large language models into running military strategy or into hiring decisions affecting millions) than about the use of AI that might be so smart it could catch its own errors.
Assessment: Can’t be checkedSaid
“But bluntly, uncontrollable superintelligence is by no means the only risk from AI, and from a policy perspective may not be the most important risk, given both the lowish (?) probability and the time horizon.”
What we tested: Uncontrollable superintelligence is by no means the only risk from AI and, from a policy perspective, may not be the most important risk, given both its lowish (?) probability and the time horizon.
Assessment: Can’t be checkedSaid
“I do think there is a lot of risk that AI will cause chaos.”
What we tested: There is a substantial risk that AI will cause chaos.
Assessment: Can’t be checkedSaid
“no matter what you feel about that p(doom | superintelligence), whether you see it as sci-fi or not, you should be worried about p(catastrophe | widespread AI adoption)”
What we tested: Whatever one thinks about doom from superintelligence, people should worry about the probability of catastrophe given widespread AI adoption.
Assessment: Can’t be checkedSaid
“The risks of current AI (bias, defamation, cybercrime, wholesale disinformation, etc) are already starting to be well documented and may themselves quickly escalate, and could lead to geopolitical instability even over the next few years.”
What we tested: Documented risks of current AI such as bias, defamation, cybercrime and disinformation may escalate quickly and could cause geopolitical instability within the next few years.
Assessment: Can’t be checkedSaid
“And what really gets the goat of people like Timnit Gebru and Abeba Birhane, not unreasonably, is that there is a good chance that any immediate effects will disproprotionately affect the dispossessed; they are very likely right about that.”
What we tested: Marcus says Gebru and Birhane are very likely right that there is a good chance any immediate effects of AI will disproportionately affect the dispossessed.
Assessment: Can’t be checkedSaid
“It’s a story for another day, but clearly you can have catastrophe without mass death, and we are heading in that direction.”
What we tested: AI can bring catastrophe without mass death, and society is heading in that direction.
Assessment: UnresolvedSaid
“We aren’t going to be able calculate p(doom) to three decimal places.”
What we tested: No one will be able to calculate p(doom) precisely, to three decimal places.
Assessment: SupportedSaid
“this too seems like a rather large assumption that is far from proven, a possibility but not demonstrably a default,”
What we tested: Yudkowsky's claim that AGI will kill everyone by default is far from proven; it is possible but not shown to be the default.
What happened: The authors explicitly limit the inference from their formal results to hypothetical superintelligent agents, noting differences between optimal and learned policies. This supports an evidential gap in a key step toward default extinction, although it does not by itself evaluate every possible extinction argument. (arXiv; Alexander Matt Turner and coauthors, )
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: Much of the risk from AI does not come from uncontrollable superintelligence.
- Attribution is uncertain