Covert racism in LLMs
We recorded 8 claims from this post. 1 was checked against evidence. 7 can’t be checked. 4 more are held. Read the post at its source.
Claims from this post (8)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
“On overt measures, the systems were fine. On covert measures, they were a disaster:”
What we tested: LLMs tested looked fine on overt racism measures but did very badly on covert racism measures when prompted with African American English.
What happened: The abstract contrasts strongly negative dialect-based stereotypes with more positive overt stereotypes. This retrospectively corroborates the findings already reported in the March 1 preprint; the event date identifies that original reporting, not a new August experiment. (Nature, )
Assessment: Can’t be checkedSaid
“The potential consequences on the world, given the widespread adoption of LLMs, are massive:”
What we tested: Because LLMs are so widely adopted, the potential consequences of their covert racism for the world are massive.
Assessment: Can’t be checkedSaid
“LLMs are, as I have been trying to tell you, too stupid to understand concepts like people and race; their fealty to superficial statistics drives this horrific stereotyping .”
What we tested: LLMs cannot understand concepts like people and race, and their reliance on superficial statistics causes racist stereotyping.
Assessment: Can’t be checkedSaid
“Counterparts in other nations should consider doing the same.”
What we tested: Regulators outside the US should consider investigating LLM makers and demanding user logs.
Assessment: Can’t be checkedSaid
“What Hofman and his collaborators (including the MacArthur Fellow Dan Jurafsky) have documented may already be having real-world impact.”
What we tested: The covert racism documented in LLMs may already be having real-world impact.
Assessment: Can’t be checkedSaid
“I would encourage Congress, the EEOC, HUD, the FTC, and others to investigate, and demand user logs and interaction data from the major LLM manufacturers.”
What we tested: Congress, the EEOC, HUD, the FTC and others should investigate covert racism in LLMs and demand user logs and interaction data from the major LLM makers.
Assessment: Can’t be checkedSaid
“And Google’s debacle shows that guardrails are never easy..”
What we tested: Google's recent debacle shows that building guardrails for LLMs is never easy.
Assessment: Can’t be checkedSaid
“But the LLM companies should recall their systems until they can find an adequate solution.”
What we tested: LLM companies should recall their systems until they find an adequate solution to covert racism.
Held from this post (4)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: Scaling up LLMs makes their covert racism worse.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: In many ways it is unknown how LLMs are actually used in real-world decisions such as housing, loans, and criminal proceedings, but there is now strong evidence to suspect covert racism to the extent they are used in such cases.
- The record does not meet the current rules. Our reading, not assessed: Trivial dialect choices in prompt wording appear to drive much of the covert racism in LLMs.