Scientists, please don’t let your chatbots grow up to be co-authors
We recorded 15 claims from this post. 7 were checked against evidence. 8 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (15)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
“Statistical word prediction is no substitute for real math, but more than a few people have thought that it was.”
What we tested: Statistical word prediction cannot substitute for real mathematical computation.
What happened: Wolfram presents direct comparisons in which ChatGPT’s plausible mathematical answers fail checks against computational results. These support the narrow claim that unaided contemporary output could not be treated as dependable calculation. (Stephen Wolfram, )
Assessment: SupportedSaid
“ChatGPT hasn’t done the conversion correctly (answer below), but the will to believe in this great new tool, full of precision (down to the second!), but lacking in reliability, is so strong that the author didn’t bother to check.”
What we tested: ChatGPT got a pace unit conversion wrong and lacks reliability, though it looks precise.
What happened: This contemporary transcription identifies the erroneous answer. Applying the NIST definitions gives 420.038784 seconds per mile, which rounds to 7:00 rather than 6:52. (Az, )
Assessment: Can’t be checkedSaid
“The worst thing about ChatGPT’s close-but-no-cigar answer is not that it’s wrong. It’s that it seems so convincing.”
What we tested: The worst thing about ChatGPT's wrong answer in the cited tweet is not that it is wrong but that it seems so convincing.
Assessment: UnresolvedSaid
“We are going to see a lot of this – misuses of ChatGPT in which people trust it, and even trumpet it—when it’s flat-out wrong.”
What we tested: There will be many cases of people trusting and promoting ChatGPT outputs that are flat-out wrong.
Assessment: SupportedSaid
“People are starting to treat ChatGPT as if it were a bona fide, well-credentialed scientific collaborator.”
What we tested: People are starting to treat ChatGPT as a credentialed scientific collaborator.
What happened: The paper lists ChatGPT and Zhavoronkov as authors and explicitly explains the decision to credit ChatGPT. The contribution statement confirms that the human author accepted its substantive arguments. (Oncoscience / Impact Journals, )
Assessment: Can’t be checkedSaid
“I sincerely hope this trend won’t last.”
What we tested: The trend of listing ChatGPT as a co-author should not continue.
Assessment: Can’t be checkedSaid
“You wouldn’t make your spell-checker or your spreadsheet a co-author.”
What we tested: ChatGPT should not be made a co-author, just as you wouldn't make your spell-checker or spreadsheet a co-author.
Assessment: Can’t be checkedSaid
“ChatGPT is a tool, but not a person. It’s at best, more like a spell-checker, or a grammar-checker, or a stats package, than it is like a scientist.”
What we tested: ChatGPT is a tool, closer to a spell-checker or stats package than to a scientist.
Assessment: UnresolvedSaid
“It is not contributing genuine ideas, nor coming up with carefully controlled experimental designs, nor summing up prior literature in thoughtful ways.”
What we tested: ChatGPT does not contribute genuine ideas, design controlled experiments, or thoughtfully summarize literature.
Assessment: UnresolvedSaid
“ChatGPT simply cannot reason well enough to to do this; as I have documented in previous essays, it has demonstrably misunderstood physics, biology, psychology, medicine, and so forth.”
What we tested: ChatGPT cannot reason well enough to make a substantive scientific contribution and has misunderstood physics, biology, psychology, and medicine.
Assessment: Can’t be checkedSaid
“Given ChatGPT’s current understanding level of scientific understanding, assigning authorship to it is disrespectful to the scientific process itself.”
What we tested: Given its current scientific understanding, assigning authorship to ChatGPT disrespects the scientific process.
Assessment: Can’t be checkedSaid
“If you declare it as a co-author, it says you are more interested in being on trend, than in being on target.”
What we tested: Declaring ChatGPT a co-author says that you are more interested in being on trend than in being on target.
Assessment: SupportedSaid
“ChatGPT has proven itself to be both unreliable and untruthful. It makes boneheaded arithmetical errors, invents fake biographical details, bungles word problems, defines non-existent scientific phenomenon, stumble over arithmetic conversion, and on and on.”
What we tested: ChatGPT is unreliable and untruthful: it makes arithmetic errors, fabricates biographical details, bungles word problems, and invents scientific phenomena.
What happened: The dated transcript records a false elementary arithmetic assertion and inconsistent calculations. It also contains correct answers, so it establishes fallibility rather than universal failure. (Vipul Naik, LessWrong; GreaterWrong mirror, )
Assessment: Can’t be checkedSaid
“If you treat ChatGPT as a sentient being, when it clearly is not, you are misleading the public into thinking ChatGPT is sentient; rather than communicating good science, you are hyping bullshit. Just stop.”
What we tested: ChatGPT is clearly not sentient, and scientists should stop treating it as such because doing so misleads the public.
Assessment: Can’t be checkedSaid
“Friends don’t let friends co-author with ChatGPT.”
What we tested: Scientists should not co-author papers with ChatGPT.
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.