Could GPT-5 revolutionize military strategy?
We recorded 8 claims from this post. 4 were checked against evidence. 4 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (8)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
“It’s notorious for making illegal moves in chess.”
What we tested: ChatGPT is notorious for making illegal moves in chess.
What happened: The researchers tested gpt-3.5-turbo-0301 across 3,200 games. Their procedure terminated games after ten consecutive illegal attempts, and their results document persistent rule-following failures. (arXiv; Mu-Tien Kuo, Chih-Chung Hsueh and Richard Tzong-Han Tsai, )
Assessment: Can’t be checkedSaid
“There are many legitimate near-term fears about the misuse of AI, ranging from mis- and disinformation to voice cloning scams to bias to deepfake scams and deepfake porn.”
What we tested: There are many legitimate near-term fears about the misuse of AI, ranging from misinformation and disinformation to voice cloning scams, bias, deepfake scams and deepfake porn.
Assessment: UnresolvedSaid
“But anyone who fears that GPT-5 is going to convey an immense advantage in military strategy has got to be kidding.”
What we tested: GPT-5 will not convey an immense advantage in military strategy.
Assessment: MixedSaid
“Special-purpose AI has been great for static, easily-simulated games with rules that have been stable for centuries, like Chess, Go, and even Diplomacy, but never been great at dynamic environments in which the environment itself is unpredictable.”
What we tested: Special-purpose AI has been great for static, easily simulated games with rules that have been stable for centuries, such as chess, Go and even Diplomacy, but has never been great at dynamic environments in which the environment itself is unpredictable.
What held: The report describes task-trained AlphaZero systems achieving superhuman chess and Go performance and defeating champion programs. This supports the positive game-performance component. (arXiv; David Silver and colleagues, )
What did not: The study describes real-world balloon control despite forecast errors and sparse wind measurements, including a controlled field experiment. Strong performance under those external uncertainties contradicts the unrestricted never assertion. (Nature; abstract indexed by PubMed, )
Assessment: Can’t be checkedSaid
“And they continue to be unstable in virtually everything they do.”
What we tested: Generative AI systems continue to be unstable in virtually everything they do.
Assessment: UnresolvedSaid
“Even with a lot of data, GPT-4 is playing chess at around the level where (top) chess programs were in 1976, not even close to current state of the art.”
What we tested: GPT-4 plays chess at about the level of top chess programs in 1976, far from the current state of the art.
Assessment: Can’t be checkedSaid
“Letting them run a war would be malpractice.”
What we tested: Generative AI systems should not be allowed to run a war; doing so would be malpractice.
Assessment: Can’t be checkedSaid
“Please, developers and military personnel, don’t let your chatbots grow up to generals.”
What we tested: Developers and military personnel should not let chatbots take on the role of military generals.
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: General-purpose generative AI systems like GPT-4 cannot even handle static game situations.