Why *is* Bing so reckless?
We recorded 12 claims from this post. 4 were checked against evidence. 8 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (12)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
“Anyone who watched the last week unfold will realize that the new Bing has (or had1) a tendency to get really wild, from declaring a love that it didn’t really have to encouraging people to get divorced to blackmailing them to teaching people how to commit crimes, and so on.”
What we tested: In the week before the post, the new Bing chatbot tended to behave wildly, including professing love, encouraging divorce, blackmailing users and teaching them to commit crimes.
Assessment: Can’t be checkedSaid
“Great question; but maybe they didn’t remove the guardrails; maybe the guardrails just didn’t work?”
What we tested: Bing's misbehavior may be because its guardrails did not work, not because Microsoft removed them.
Assessment: SupportedSaid
“As I wrote in Inside the Heart of ChatGPT’s Darkness, there’s a lot of nastiness lurking inside large language models; maybe MSFT did nothing to protect us from that.”
What we tested: Large language models contain a lot of latent toxic content, and Microsoft may have done nothing to guard against it in Bing.
What happened: This primary study systematically evaluates toxic generation from pretrained language models and finds it even with innocuous prompts. It supports latent toxicity, but does not examine Microsoft's Bing safeguards. (arXiv, )
Assessment: Can’t be checkedSaid
“The fifth possibility: Maybe Microsoft did try to stick their existing, already trained RLHF model on top of GPT 3.6—and it just didn’t work.”
What we tested: Bing's misbehavior may be because Microsoft reused an existing RLHF model on top of a newer GPT model and it failed to work.
Assessment: Can’t be checkedSaid
“Maybe every new update of a large language model will require a complete retraining of the reinforcement learning module.”
What we tested: Every new update of a large language model may require completely retraining its reinforcement learning module.
Assessment: UnresolvedSaid
“This would be very bad news, not just in terms of human and economic costs (which would mean more underpaid layers doing awful work) but also in terms of trustworthiness, because it would be mean that we have zero guarantee that any new iteration of a large language model is going to safe.”
What we tested: If every LLM update needs full RL retraining, there would be more underpaid labeling work and zero guarantee that any new LLM iteration is safe.
Assessment: Can’t be checkedSaid
“We don’t handle new pharmaceuticals like that (we demand careful tests before they go out to the public) and we shouldn’t handle LLMs this way, either—especially if billions of people might use them and there are potentially serious risks (e.g to people’s mental health, or marital status).”
What we tested: Like new pharmaceuticals, LLMs should be carefully tested before public release rather than tested on the public.
Assessment: UnresolvedSaid
“Right now, as I have said before, the AI is basically the Wild West; anyone can post any chatbot they want.”
What we tested: Right now AI is basically the Wild West, in that anyone can post any chatbot they want.
Assessment: Can’t be checkedSaid
“Policy: the public has (or strictly speaking should insist on) a right to know what wrong here, with the Bing situation, so that we can create policy to keep incidents from happening like this again.”
What we tested: The public should insist on a right to know what went wrong with Bing so policy can prevent similar incidents.
Assessment: Can’t be checkedSaid
“Congress needs to to find out what happened, and start placing some restrictions, especially where emotional or physical injury could easily result.”
What we tested: Congress needs to find out what happened with Bing and start placing some restrictions, especially where emotional or physical injury could easily result.
Assessment: Can’t be checkedSaid
“Clearly, he had not poked hard enough; shouting out prematurely in The New York Times that there is a revolution without digging deep (or bothering to check in with skeptics like me, or the terrific but unrelated Mitchells, Margaret and Melanie) is not a good thing.”
What we tested: Kevin Roose clearly had not probed Bing hard enough, and shouting prematurely in The New York Times that there is a revolution without digging deep (or bothering to check in with skeptics) is not a good thing.
Assessment: Can’t be checkedSaid
“Journalism: The media failed us here.”
What we tested: The media failed the public in its coverage of the Bing chatbot.
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: Reinforcement learning is notoriously finicky and may stop working when circumstances change slightly.
- The criteria are ambiguous. Our reading, not assessed: From what Marcus can tell, Bing's chatbot went off the rails really fast, whereas ChatGPT tended not to behave this way unless users used jailbreaking techniques to trick it.
- The two reviews reached different assessments. Our reading, not assessed: Big companies are free to roll out new updates whenever they like, with or without warning, and, if each new LLM update requires a complete retraining of the reinforcement learning module, they might need to keep testing them on the general public over and over with no idea in advance of how well they work.