Where we are right now on open source and potential AI risk.
We recorded 4 claims from this post. 1 was checked against evidence. 3 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (4)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
“Nobody knows for SURE that there are no serious possible consequences of open source AI, e.g in terms of bioweapons, misinformation generation etc, as Séb Krier at DeepMind nicely highlights above.”
What we tested: No one knows for certain that open source AI has no serious possible consequences, such as for bioweapons or misinformation generation.
What happened: The researchers explicitly describe unresolved protection against misuse and report experiments undoing safety training in publicly available Llama 2 models. This supports the absence of an established safety guarantee, without proving that serious real-world harm occurred. (arXiv; Simon Lermen, Charlie Rogers-Smith and Jeffrey Ladish, )
Assessment: Can’t be checkedSaid
“There is a nonzero chance that they have put us all at massive risk, with zero oversight.”
What we tested: There is a nonzero chance that the company's open source AI push has put everyone at massive risk without oversight.
Assessment: Can’t be checkedSaid
“We really ought be a bit more patient about comparing risks and benefits.”
What we tested: We ought to be a bit more patient about comparing the risks and benefits of open source AI.
Assessment: Can’t be checkedSaid
“The fact that a single company can unilaterally make this decision for all of humanity is terrifying.”
What we tested: It is terrifying that a single company can unilaterally make the open source AI decision for all of humanity.
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: The company pushed open source AI as quickly as possible, apparently without external oversight and without waiting for formal results or scientific consensus.
- The criteria are ambiguous. Our reading, not assessed: A giant company that is behind in AI, at least in public profile, is aggressively pushing open source AI, apparently to raise its profile.