Automatic Disinformation Threatens Democracy— and It’s Here
We recorded 12 claims from this post. 5 were checked against evidence. 7 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (12)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“Elections may hinge on them.”
What we tested: Elections may hinge on automatic misinformation generation systems.
Assessment: Can’t be checkedSaid
“The killer app of AGI may be killing truth.”
What we tested: The killer app of AGI may be killing truth.
Assessment: Can’t be checkedSaid
“Businesses may be at a disadvantage if they too don’t use such systems, e.g., to counter adverse news, automatically.”
What we tested: Businesses that do not use automatic misinformation generation systems may be at a disadvantage.
Assessment: Can’t be checkedSaid
“The temptation for all sides to use these tools is likely to be overwhelming.”
What we tested: The temptation for all political sides to use automatic misinformation tools is likely to be overwhelming.
Assessment: UnresolvedSaid
“Automatic tools like these will surely accelerate the enshittification of the internet that I discussed yesterday.”
What we tested: Automatic misinformation generation tools will accelerate the degradation of the internet.
Assessment: UnresolvedSaid
“It’s a powerful reminder that our existing laws are not sufficient to protect us from the full scope of harms that generative AI is beginning to enable .”
What we tested: Existing laws are not sufficient to protect against the full scope of harms generative AI is beginning to enable.
Assessment: Can’t be checkedSaid
“We ought to demand that any content like this that is automatically generated be labeled (no existing law does that) and watermarked (no existing law does that either)”
What we tested: We ought to demand that any automatically generated content like this (the CounterCloud-style misinformation discussed) be labeled and watermarked, and no existing law requires either.
Assessment: Can’t be checkedSaid
“we ought to demand that the wholesale generation of misinformation be punishable; there ought to be enforcement.”
What we tested: Wholesale generation of misinformation should be punishable and enforced against.
Assessment: SupportedSaid
“It highlights the inadequacy of the voluntary agreements such as the one that 7 tech companies recently made with The White House. Those agreements called for watermarking, but because they are voluntary, they do nothing to prevent bad actors from rolling their own automated misinformation software.”
What we tested: Because the White House AI agreements with 7 tech companies are voluntary, they are inadequate and do nothing to stop bad actors building their own automated misinformation software.
What happened: The full document assigns undertakings to participating companies and contains no obligation on independent nonparticipants. It also includes safeguards against misuse and unauthorized access, which could provide indirect benefits without binding outside developers. (The White House, publishing participating companies' commitments, )
Assessment: SupportedSaid
“Social media companies better watch out, too; they will be polluted in much the same way as the internet as a whole.”
What we tested: Social media platforms will be polluted by automated misinformation much as the internet as a whole will be.
What happened: The report documents multiple covert campaigns using generated material on Telegram, X, Facebook, Instagram, and websites, including Doppelganger and Zero Zeno. These establish repeated distribution through the mechanism described, while the report also says AI had not meaningfully increased audience reach. (OpenAI, )
Assessment: UnresolvedSaid
“We have no such thing; building it would require a major reshaping of research directions, towards an AI based in facts and competent in reasoning, rather than just the powerful tools for text prediction that we have now.”
What we tested: We have no AI smart enough to know when it is being used to spew falsehood, and building it would require a major reshaping of research directions toward AI based in facts and competent in reasoning, rather than just the powerful text-prediction tools we have now.
Assessment: Can’t be checkedSaid
“What we really need is AI that is smart enough to know when it is being used to spew falsehood.”
What we tested: We need AI smart enough to know when it is being used to spread falsehoods.
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.