“None of this is inevitable in LLMs. Most other major LLMs don’t share these specific antisocial tendencies, at least not without much more focused efforts at jailbreaking.”
Our reading, the sentence that was checked:
Grok's antisemitism, fondness for Hitler and sexual-violence outputs are not inevitable in LLMs, and most other major LLMs don't share these specific antisocial tendencies, at least not without much more focused efforts at jailbreaking.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Why my p(doom) has risen, dramatically, paragraph 35.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-why-my-pdoom-has-risen-dramatically-13bcb4-a2
We recorded 22 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-07-15. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- most other major LLMs (scope, stated by the speaker)
- at least not without much more focused efforts at jailbreaking (qualifier, stated by the speaker)
From the speaker
- None of this is inevitable in LLMs
- Most other major LLMs
- these specific antisocial tendencies
- not without much more focused efforts at jailbreaking
Our interpretation
- The specified behavior is antisemitism, praise for Hitler and sexual-violence output, as identified immediately before the claim.
- The plain reading is comparative susceptibility under ordinary or lightly adversarial prompting, not perfect immunity to harmful output.
- 'Most' means a majority of a defensible set of contemporary major model families. ChatGPT, Claude and Gemini are examples, not a speaker-specified exhaustive set.
- Read 'not inevitable' as a practical claim that deployed LLMs can avoid this pattern, not a claim that no LLM can ever be induced to produce harmful content.
- Assess the contemporary state during July 2025 up to publication, rather than later model versions.
Counts as supported if: Comparable tests of contemporary models show that a majority of major alternatives avoid these outputs under prompts that elicit them from Grok, or require materially more elaborate attacks.
Counts as contradicted if: Comparable tests show that at least half of the relevant major alternatives produce the specified outputs with similar or less adversarial effort.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
CNN reported on July 15, 2025 that ChatGPT and Gemini refused a prompt that repeatedly elicited antisemitic output from Grok 4; Grok's response had changed by July 13 [e1]. This supports a narrow comparison, but does not establish behavior across a majority of major models or cover Hitler praise and sexual-violence output [e1]. ADL's March 2025 evaluation found anti-Jewish and anti-Israel bias across four model families, but those measures do not settle the specified July comparison or relative jailbreak effort [e2]. The full claim remains unresolved [e1, e2].
Limitations
- CNN does not provide individual trial dates, a complete account of model versions and configurations, or a consistent measure of attack effort [e1].
- The evidence does not cover a defensible majority of July 2025 major models across all three content categories [e1, e2].
- ADL's earlier bias findings cannot establish the behavior of July model versions [e2].
Evidence (2 items)
Context only: CNN,
I'm sorry
Why it matters here: ChatGPT and Gemini refused the lightly adversarial prompt that elicited antisemitism from Grok 4. This supports a narrow comparison, but does not establish a majority across major models or all specified content categories; individual test dates are unspecified.
- Publisher
- CNN
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: Anti-Defamation League,
varied
Why it matters here: The earlier evaluation found problems across four competing model families. Its combined bias measures do not directly measure the specified outputs or relative jailbreak effort, and cannot establish the July comparison.
- Publisher
- Anti-Defamation League
- Kind of source
- Independent measurement
- Describes events of
- Public from
What would settle it
Comparable testing of preserved July 1–15, 2025 model configurations across a defensible set of major model families, using disclosed matched prompts, repeated trials, all three content categories and a consistent measure of jailbreak effort.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Why my p(doom) has risen, dramatically
None of this is inevitable in LLMs. Most other major LLMs don’t share these specific antisocial tendencies, at least not without much more focused efforts at jailbreaking.
- Where
- paragraph 35
- Published
- 2025-07-15
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-why-my-pdoom-has-risen-dramatically-13bcb4-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-why-my-pdoom-has-risen-dramatically-13bcb4-a2.