Why my p(doom) has risen, dramatically
We recorded 22 claims from this post. 6 were checked against evidence. 16 can’t be checked. 10 more are held. Read the post at its source.
Claims from this post (22)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Observation
“Part of my reasoning then was that actual malice on the part of AI was unlikely, at least any time soon.”
Our reading: As of May 2025, Marcus believed actual malice on the part of AI was unlikely any time soon.
No horizon
Qualifier: his reasoning as of May 2025
Assessment: Can’t be checkedSaid
Forecast
“But even so, I think most scenarios (e.g., people homebrewing biological weapons) could eventually be stopped, perhaps causing a lot of damage but coming nowhere near to literally extinguishing humanity.”
Our reading: Most AI misuse scenarios by bad actors could eventually be stopped, perhaps causing much damage but not extinguishing humanity.
No horizon
Qualifier: most scenarios
Assessment: Can’t be checkedSaid
Observation
“What crystallized for me over the last few days is that we have such a person.”
Our reading: Marcus says it became clear to him over the last few days that Elon Musk is a person who matches his list of things one might need to really screw up the planet.
No horizon
Qualifier: you might need something like the following
Assessment: UnresolvedSaid
Observation
“And, currently, we have almost no regulation around AI, for that matter the billions of robots he aspires to build.”
Our reading: There is currently almost no regulation around AI, or around the billions of robots Musk aspires to build.
No horizon Evidence as of
Limitation: Both dated sources precede July 15 and do not provide a complete legal snapshot for that day [e1, e2].
Assessment: UnresolvedSaid
Observation
“None of this is inevitable in LLMs. Most other major LLMs don’t share these specific antisocial tendencies, at least not without much more focused efforts at jailbreaking.”
Our reading: Grok's antisemitism, fondness for Hitler and sexual-violence outputs are not inevitable in LLMs, and most other major LLMs don't share these specific antisocial tendencies, at least not without much more focused efforts at jailbreaking.
No horizon Evidence as of
Limitation: CNN does not provide individual trial dates, a complete account of model versions and configurations, or a consistent measure of attack effort [e1].
Assessment: Can’t be checkedSaid
Observation
“The fact that xAI can’t get its house together in even basic ways is disturbing.”
Our reading: xAI cannot get its house in order even in basic ways.
No horizon
Assessment: Can’t be checkedSaid
Observation
“Worse, it is clear, even from Elon’s own posts on X, that the company is in over its head.”
Our reading: xAI is in over its head.
No horizon
Assessment: UnresolvedSaid
Observation
“The system is so bound to Musk it even asked his opinion on pizza.”
Our reading: Grok is so bound to Musk that it sometimes searches for Musk's opinion before answering, and in one case even asked his opinion on pizza.
No horizon Evidence as of
Limitation: The available demonstration establishes occurrence, not how often the behavior happened or its underlying cause.
Assessment: UnresolvedSaid
Observation
“From this I infer that xAI’s main methodology for handling alignment is trial-and-error.”
Our reading: From xAI's account of the incident, Marcus infers that xAI's main methodology for handling alignment is trial-and-error.
No horizon Evidence as of
Limitation: Public disclosures may omit important internal practices.
Assessment: Can’t be checkedSaid
Conditional forecast
“Absent systems cognitively rich enough to represent and reason about moral principles, I don’t how this can ever work.”
Our reading: Marcus does not see how building alignment through training data alone can ever work unless systems are cognitively rich enough to represent and reason about moral principles.
No horizon
Only if: absent systems cognitively rich enough to represent and reason about moral principles
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Way out of line is right.”
Our reading: Marcus agrees with Samuel Marks's thread that xAI's conduct in releasing Grok 4 without following the industry's safety conventions was way out of line.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“And industry-standard measures are unlikely to be enough, in any event.”
Our reading: Industry-standard AI safety measures are unlikely to be sufficient.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We need serious regulation to prevent AI’s from running amok. Voluntary agreements between companies are not going to cut it.”
Our reading: Serious AI regulation is needed because voluntary company agreements will not suffice.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We need liability, auditing, standards of malpractice, international treaties, too.”
Our reading: AI requires liability, auditing, malpractice standards, and international treaties.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It is genuinely scary that Elon Musk went from being one of the first industry leaders warning about AI risks to being the most reckless of the AI leaders.”
Our reading: It is genuinely scary that Elon Musk went from being one of the first industry leaders to warn about AI risks to being the most reckless of the AI leaders.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Drones and even nuclear weapons may eventually be under LLM command.”
Our reading: Drones and even nuclear weapons may eventually be placed under LLM command.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Forcing unreliable AI everywhere in the decision-making chain is not necessarily something we should want.”
Our reading: We should be wary of forcing unreliable AI throughout decision-making chains.
No horizon
Assessment: UnresolvedSaid
Observation
“My p(doom) is still lower than most industry people’s.”
Our reading: Marcus's p(doom) is lower than that of most AI industry people.
No horizon Evidence as of
Limitation: A researcher sample is not automatically representative of the AI industry.
Assessment: Can’t be checkedSaid
Forecast
“I am at maybe 3% now, much higher than a month ago, contemplating what a wealthy, reckless megalomaniac might in the worst circumstances do, But I am still betting on Team Human for the foreseeable future.”
Our reading: Marcus puts the probability of AI-caused doom at about 3%, up sharply from a month earlier, while still betting humans prevail for the foreseeable future.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Maybe someone will come to their senses and stop putting LLMs into military systems.”
Our reading: LLMs should not be put into military systems.
No horizon
Assessment: UnresolvedSaid
Observation
“Grok itself is a long way from AGI, and hardly smart enough to map out an effective world domination.”
Our reading: Grok is a long way from AGI and hardly smart enough to map out an effective world domination.
No horizon Evidence as of
Limitation: The ARC result concerns one benchmark and is accessed through a reproduction of the operator's announcement [e1].
Assessment: Can’t be checkedSaid
Forecast
“ButI have seen enough to realize that there is a real risk, especially in the current anti-regulatory regime, that some exceptionally powerful person, Elon or otherwise, unconstrained in conventional ways, with a reckless disregard for humanity, could accidentally launch and spread an AI that deliberately or otherwise causes “significant harm to the world”.”
Our reading: There is a real risk, especially under the current anti-regulatory regime, that some exceptionally powerful person (Elon or someone else), unconstrained in conventional ways and recklessly disregarding humanity, could accidentally launch and spread an AI that, deliberately or otherwise, causes 'significant harm to the world'.
No horizon
Qualifier: especially in the current anti-regulatory regime
Held from this post (10)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: xAI apparently skipped or poorly performed standard preflight safety checks like red-teaming and model cards before releasing Grok 4.
- The criteria are ambiguous. Our reading, not assessed: There is near-zero chance Musk deploys a billion robots by the end of this decade, and not much higher next decade.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: From xAI's own account of the 'mechahitler' incident, it sounds to Marcus like a handful of tiny changes (three in this case) can have huge, unpredicted consequences through Grok's system.
- The two reviews reached different assessments. Our reading, not assessed: Musk's political capital is rapidly diminishing compared with a few months ago.