Two models of AI oversight - and how things could go deeply wrong
We recorded 20 claims from this post. 5 were checked against evidence. 15 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (20)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“What I am worried about is regulatory capture; governments making rules that entrench the incumbents whilst doing too little for humanity.”
What we tested: There is a risk that governments make AI rules that entrench incumbents while doing too little for humanity.
Assessment: Can’t be checkedSaid
“If big tech writes the rules, without outside input, we are unlikely to wind up with the right rules.”
What we tested: If big tech writes AI rules without outside input, the right rules are unlikely to result.
Assessment: Can’t be checkedSaid
“Putting it bluntly: if we have the right regulation; things could go well. If we have the wrong regulation, things could badly.”
What we tested: With the right AI regulation things could go well; with the wrong regulation things could go badly.
Assessment: Can’t be checkedSaid
“We still have agency here; we can still, I think, build a very positive AI future.”
What we tested: It is still possible to build a very positive AI future.
Assessment: Can’t be checkedSaid
“Press releases and photo opportunities that highlight governments hanging out with the tech moguls they seek to regulate, without independent voices in the room, send entirely the wrong message.”
What we tested: Government photo opportunities with tech moguls without independent voices send the wrong message.
Assessment: Can’t be checkedSaid
“But much depends on how much the government stands up to big tech, and a lot of that depends on having independent voices – scientists, ethicists, and representatives of civil society – at the table.”
What we tested: The AI future depends largely on government standing up to big tech, which depends on independent voices being at the table.
Assessment: SupportedSaid
“But their current, actual products are definitely not transparent, and at least in some ways, demonstrably not safe.”
What we tested: Microsoft's current AI products are not transparent and in some ways demonstrably not safe.
What happened: The researchers describe Bing as a system they could test without access to its internals. They identify GPT-4 as its underlying model and distinguish available high-level descriptions from unavailable implementation details. (Kai Greshake and colleagues / arXiv, )
Assessment: SupportedSaid
“Bing relies on GPT-4, and we (eg, in the scientific community) don’t have access to how GPT-4 works, and we don’t have access to what data its trained on (vital, since we know that systems can bias eg political thought and hiring decisions based on those undisclosed data) — that’s about as far away from transparency as we could be.”
What we tested: Scientists lack access to how GPT-4 works and its training data, making it about as far from transparent as possible.
What happened: The contemporaneous complaint identifies the specific missing disclosures. Its statement is corroborated by OpenAI's own report; the complaint itself is not an adjudicated regulatory finding. (Center for AI and Digital Policy, )
Assessment: SupportedSaid
“We also know, for example, that Bing has defamed people, and it has misread articles as saying the opposite of what they actually say, in service of doing so.”
What we tested: Bing has defamed people, and in doing so has misread articles as saying the opposite of what they actually say.
What happened: The same post identifies Turley's USA Today rebuttal among Bing's sources for the false allegation. It directly connects the reputational falsehood with the reversed source meaning. (Will Oremus, reporting on Twitter, )
Assessment: UnresolvedSaid
“I don’t see any reason to think these plugins are in fact safe. (They are far less sandboxed and less rigorously controlled than Apple app store apps.)”
What we tested: Marcus says he does not see any reason to think ChatGPT plugins are in fact safe, noting that they are far less sandboxed and less rigorously controlled than Apple App Store apps.
Assessment: SupportedSaid
“Meanwhile, ChatGPT plugins (produced by OpenAI, which they have a close tie with) open a wide range of security problems: those plugins can access the internet, read and write files, and impersonate people (e.g., to phish for credentials), all alarms to any security professional.”
What we tested: ChatGPT plugins open a wide range of security problems, since they can access the internet, read and write files, and impersonate people (e.g., to phish for credentials), all of which would alarm any security professional.
What happened: The launch documentation establishes browsing and file-processing capabilities and reports fraudulent-email and prompt-injection findings without safeguards. It also describes sandboxing, network controls, and a GET-only browser, which limit the claim's scope. (OpenAI, )
Assessment: Can’t be checkedSaid
“This is where the government needs to step up and say “transparency and safety are indeed requirements; you’ve flouted them; we won’t let you do that anymore.””
What we tested: Government should require transparency and safety from AI companies and stop them from flouting those requirements.
Assessment: Can’t be checkedSaid
“We don’t need more photo opportunities, we need regulation – with teeth.”
What we tested: AI needs enforceable regulation rather than photo opportunities.
Assessment: Can’t be checkedSaid
“More broadly, at an absolute minimum governments need to establish an approval process for any AI that deployed at large scale, showing that the benefits outweigh the risks, and to mandate post-release auditing - by independent outsiders - of any large-scale deployments.”
What we tested: At an absolute minimum, governments need to establish an approval process for any AI deployed at large scale that shows the benefits outweigh the risks, and need to mandate post-release auditing of any large-scale deployment by independent outsiders.
Assessment: Can’t be checkedSaid
“And governments need to make sure that there are strong liability laws in place, to ensure that if the big tech companies cause harm with their products, they be held responsible.”
What we tested: Governments need to make sure strong liability laws are in place so that, if big tech companies cause harm with their products, they are held responsible.
Assessment: Can’t be checkedSaid
“Goverments should demand that systems only use copyrighted content from content providers that opt-in, and that all machine-generated content be labeled as such.”
What we tested: Governments should require AI systems to use copyrighted content only from opt-in providers and to label all machine-generated content.
Assessment: Can’t be checkedSaid
“Letting the companies set the rules on their own is unlikely to get us any of these places.”
What we tested: If AI companies set the rules on their own, the needed safeguards are unlikely to result.
Assessment: Can’t be checkedSaid
“I actually think that he is at least in part sincere, that it is not just a ploy to keep the incumbents in and small competitors out, that he is genuinely worried about the risks (ranging from misinformation, to serious physical harm to humanity).”
What we tested: Marcus thinks Sam Altman is at least in part sincere in asking for AI regulation, that it is not just a ploy to keep incumbents in and small competitors out, and that he is genuinely worried about AI risks ranging from misinformation to serious physical harm to humanity.
Assessment: Can’t be checkedSaid
“What really matters is what governments around the world come up with by way of regulation.”
What we tested: What really matters is the regulation that governments around the world come up with.
Assessment: Can’t be checkedSaid
“We would never leave the pharmaceutical industry to entirely self-regulate itself, and we shouldn’t leave AI to do so, either.”
What we tested: The AI industry should not be left to regulate itself, just as pharma is not.
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: Microsoft reportedly has access to all OpenAI models and can race forward with them regardless of Altman's concerns.
- The two reviews read the claim differently