Bravo @ElonMusk for wanting Twitter to become the world’s “most accurate source of information”
We recorded 9 claims from this post. 6 were checked against evidence. 3 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (9)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
“The problem is about to get much, much worse.”
What we tested: The problem of misinformation on Twitter is about to get much, much worse.
Assessment: SupportedSaid
“Knockoffs of GPT-3 are getting cheaper and cheaper, which means that the cost of generating misinformation is going to zero, and the quantity of misinformation is going to rise—probably exponentially.”
What we tested: Knockoffs of GPT-3 are getting cheaper and cheaper.
What happened: The release compares optimized and baseline implementations of open text generators, including BLOOM and OPT, using dollars per million generated tokens. Its cost-sensitive benchmarks report substantial reductions, directly supporting cheaper attainable inference before the post. (Microsoft DeepSpeed, )
Assessment: SupportedSaid
“Knockoffs of GPT-3 are getting cheaper and cheaper, which means that the cost of generating misinformation is going to zero, and the quantity of misinformation is going to rise—probably exponentially.”
What we tested: Cheaper GPT-3 knockoffs will drive the cost of generating misinformation toward zero.
What happened: The experiments establish that alternatives can produce convincing election-disinformation content. For ten articles daily over a month, the cost section estimates $0.30 using Gemini or $9 using hosted Llama 3 70B, compared with $4,500 for the cited human-writing service; these are generation-stage estimates, not total campaign budgets. (PLOS ONE, )
Assessment: UnresolvedSaid
“As the pace of machine-generated misinformation picks up, Twitter’s existing effort, Community Notes (aka Birdwatch), which is mostly done manually, by humans, is going to get left in the dust.”
What we tested: As machine-generated misinformation accelerates, Twitter's mostly manual Community Notes will be overwhelmed.
Assessment: Can’t be checkedSaid
“No matter how much data you can collect, don’t count on deep learning.”
What we tested: Twitter should not count on deep learning to detect misinformation, regardless of how much data it collects.
Assessment: UnresolvedSaid
“Large language models are without equal at generating misinformation, but they suck at detecting it.”
What we tested: LLMs are unmatched at generating misinformation but poor at detecting it.
Assessment: UnresolvedSaid
“Scaling won’t solve this.”
What we tested: Scaling LLMs will not solve their inability to reason about truth and detect misinformation.
Assessment: Can’t be checkedSaid
“You are going to need AI. But not (just) the popular stuff.”
What we tested: Twitter will need AI to fight misinformation, but not just popular AI approaches.
Assessment: Can’t be checkedSaid
“You’re going to need something different.”
What we tested: Fighting misinformation on Twitter requires an AI approach different from deep learning/LLMs.
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: The quantity of misinformation will rise, probably exponentially, as cheap LLMs drive generation costs down.
- The criteria are ambiguous. Our reading, not assessed: Accuracy depends on some reference standard of truth, and large language models have no way of reasoning about truth.