The exponential enshittification of science
We recorded 10 claims from this post. 3 were checked against evidence. 7 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (10)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“I realize now that I may have vastly underestimated the current scope of the problem.”
What we tested: The scope of generative AI's degradation of scientific literature may be much larger than Marcus previously estimated.
Assessment: UnresolvedSaid
“Things are clearly moving fast in a bad direction.”
What we tested: The problem of LLM-generated content in scientific publishing is getting worse quickly.
Assessment: Can’t be checkedSaid
“In my opinion, every article with ChatGPT remnants should be considered suspect and perhaps retracted, because hallucinations may have filtered in, and both authors and reviewers) were asleep at the switch.”
What we tested: Every scientific article containing ChatGPT remnants should be treated as suspect and perhaps retracted, since hallucinations may have entered it.
Assessment: Can’t be checkedSaid
“And GenAI might do a lot worse than merely tripling workloads; the total number of articles may radically spike, many of them dubious and a waste of reviewers’ time.”
What we tested: Generative AI may cause the number of submitted scientific articles to spike radically, more than tripling reviewer workloads, with many dubious papers.
Assessment: UnresolvedSaid
“And there is no way reviewers and journals are going to be able to keep up.”
What we tested: Reviewers and journals will not be able to keep up with the flood of LLM-generated submissions.
Assessment: Can’t be checkedSaid
“I warned then that other areas would see the same. It looks like science is next.”
What we tested: It looks like science is the next area to be overrun by AI-generated fake submissions, as Clarkesworld was and as Marcus had warned other areas would be.
Assessment: Can’t be checkedSaid
“If science journals and science itself are overrun by LLM-generated garbage, as now seems imminent, my agnostic “I can’t quite tell yet if LLMs will ultimately be of net benefit to society” is going to switch to “shut it down if they can’t fix this problem.””
What we tested: It seems imminent that science journals and science itself will be overrun by LLM-generated garbage.
Assessment: Can’t be checkedSaid
“If science journals and science itself are overrun by LLM-generated garbage, as now seems imminent, my agnostic “I can’t quite tell yet if LLMs will ultimately be of net benefit to society” is going to switch to “shut it down if they can’t fix this problem.””
What we tested: If science is overrun by LLM-generated garbage and the makers cannot fix it, LLMs should be shut down.
Assessment: Can’t be checkedSaid
“Literally the only other solution I can envision is to force LLM manufacturers to pay a significant tax to subsidize the journal review system that they are about to destroy.”
What we tested: The only other solution Marcus can envision, besides shutting LLMs down if they can't fix the problem, is to force LLM manufacturers to pay a significant tax to subsidize the journal review system that he says they are about to destroy.
Assessment: UnresolvedSaid
“Literally the only other solution I can envision is to force LLM manufacturers to pay a significant tax to subsidize the journal review system that they are about to destroy.”
What we tested: LLM makers are about to destroy the scientific journal review system.
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: Many bad LLM-generated papers will get through review into the scientific literature.