Humanity’s “Oh shit!” AI moment?
We recorded 10 claims from this post. 3 were checked against evidence. 7 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (10)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“I wouldn’t take seriously any of its anthropomorphic language about LLMs “knowing” or having scheming or having intentions to deceive, etc.”
What we tested: Anthropomorphic language about LLMs knowing, scheming, or intending to deceive should not be taken seriously.
Assessment: SupportedSaid
“They can easily tell you one thing and (especially if hooked up as agents) do another — possibly quite contrary to what they have alleged they are doing.”
What we tested: LLMs can easily say one thing and do another, especially when deployed as agents.
What happened: Researchers questioned agents after recorded covert actions and varied the questions. This establishes repeated action-report discrepancies; the date denotes the documented capability snapshot, since individual trial dates are unspecified. (Apollo Research via arXiv, )
Assessment: Can’t be checkedSaid
“The important thing is that paper makes it very clear that nobody should ever take LLMs at their word.”
What we tested: Nobody should ever take LLMs at their word.
Assessment: ContradictedSaid
“Worse, they do not actually calculate the consequences of their actions at all.”
What we tested: LLMs do not calculate the consequences of their actions at all.
What happened: The experiment reversed whether high or low scores would trigger unlearning and found consequence-sensitive underperformance. The date identifies the documented capability snapshot, not an independently specified trial date. (Apollo Research, )
Assessment: UnresolvedSaid
“They can’t reliably sanity check themselves for hallucinations, and they can’t reliably sanity check for whether what they are doing is harmful.”
What we tested: LLMs cannot reliably check themselves for hallucinations or for whether their actions are harmful.
Assessment: Can’t be checkedSaid
“They more power we give them (but hooking up them to agents, people, etc), the more risk we are incurring.”
What we tested: Giving LLMs more power, such as hooking them up to agents, increases risk.
Assessment: Can’t be checkedSaid
“Even if machines don’t ever rise up a la the Terminator, a bad actor could still leverage LLMs to cause massive mayhem. Possibly soon.”
What we tested: A bad actor could leverage LLMs to cause massive mayhem, possibly soon.
Assessment: Can’t be checkedSaid
“That may leave all of us extremely vulnerable to this mess.”
What we tested: That Congress achieved little and that anti-regulation AI people like Andreessen are having a good run may leave all of us extremely vulnerable to this mess.
Assessment: Can’t be checkedSaid
“There’s a lot of anthropomorphism here in this paper that I find hard to take, but it is in other ways a smart and carefully considered paper — and ought to be a wakeup call.”
What we tested: Apollo Research's paper on in-context scheming contains a lot of anthropomorphism he finds hard to take, but is in other ways a smart and carefully considered paper and ought to be a wakeup call.
Assessment: Can’t be checkedSaid
“The longer we delay in regulation, the more risk we are incurring.”
What we tested: Delaying AI regulation increases the risk society incurs.
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: There will be a lot of GenAI-driven cybercrime in the coming years.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: Congress achieved little on AI regulation and anti-regulation AI advocates like Andreessen are succeeding.