The Rise and Fall of ChatGPT?
We recorded 7 claims from this post. 4 were checked against evidence. 3 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (7)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“Maybe it’s too soon to declare victory on my August 12th bet that ChatGPT will turn out to be a dud, but I will tell you one thing, it’s not obvious I am wrong.”
What we tested: As of late August 2023, it is not obvious that Marcus's bet that ChatGPT will turn out to be a dud is wrong, though it may be too soon to declare victory.
Assessment: SupportedSaid
“And of course not every enterprise-scale experiment will be successful.”
What we tested: Not every enterprise-scale generative AI project will succeed.
What happened: Reporting from Tuesday's bankruptcy hearing documents inability to fulfill the district contract and preparations to close the supplier. Together with the deployed chatbot's shutdown, this establishes failure of the original implementation, beyond a routine temporary interruption. (The 74, )
Assessment: SupportedSaid
“The backlash as begun.”
What we tested: A backlash against generative AI and ChatGPT had begun by August 2023.
What happened: Gioia's essay is direct evidence that prominent critical commentary existed before Marcus's post. Its claims about demand are not accepted as evidence of actual demand. (The Honest Broker / Ted Gioia, )
Assessment: Can’t be checkedSaid
“This could be particularly devastating because Large Language Models aren’t like classical databases in which individual pieces of data can be removed at will; if any content is removed, the entire model must (so far as I understand it) be retrained, at great expense.”
What we tested: The potential New York Times lawsuit could be particularly devastating for OpenAI because, unlike classical databases, LLMs cannot have individual pieces of data removed at will; as far as Marcus understands it, removing any content means retraining the entire model at great expense.
Assessment: UnresolvedSaid
“In less than a year, ChatGPT has gone from being mistaken for AGI to being the butt of a joke, and an insulting shorthand for robotic, incoherent, unreliable, and untrustworthy.”
What we tested: Within less than a year, ChatGPT went from being mistaken for AGI to being a joke and insulting shorthand for robotic, incoherent, unreliable and untrustworthy.
Assessment: UnresolvedSaid
“And the financial challenges are starting to mount up.”
What we tested: Financial challenges facing ChatGPT/OpenAI were starting to mount as of August 2023.
Assessment: Can’t be checkedSaid
“It ain’t over yet, but what a stunning reversal of fortune.”
What we tested: ChatGPT has suffered a stunning reversal of fortune, though the story is not over.
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: Long-known problems with LLMs, such as producing offensive or misguided outputs, appear to persist in the latest models, including Google's.
- The two reviews reached different assessments. Our reading, not assessed: Many companies are struggling to deploy generative AI because of high costs and confusion, and without deployment there is no revenue.