Three baffling claims about AI and machine learning in four days, statistical errors in top journals, and claims from Yann LeCun that you should not believe.
We recorded 13 claims from this post. 12 were checked against evidence. 1 can’t be checked. 4 more are held. Read the post at its source.
Claims from this post (13)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
“Every truck I know is still driven by a person (except in some limited-use test pilots), and no radiologists have yet been replaced.”
What we tested: As of October 2022, every truck the author knows of is still driven by a person (except in some limited-use test pilots), and no radiologists have yet been replaced.
Assessment: UnresolvedSaid
“Eleven years later, comprehension is still lacking (see any of my recent essays here) and very few if any jobs have actually been replaced by AI.”
What we tested: As of October 2022, AI still lacks comprehension, and very few if any jobs have actually been replaced by AI.
Assessment: Can’t be checkedSaid
“Undoubtedly clever intelligences than ours are possible and may yet materialize.”
What we tested: Intelligences cleverer than human intelligence are possible and may eventually materialize.
Assessment: UnresolvedSaid
“Human children are constantly learning new things, often from tiny amounts of data. There is literally nothing like that in the AI world.”
What we tested: No current AI system can learn new things from tiny amounts of data the way human children constantly do.
Assessment: UnresolvedSaid
“Hedging with probably makes it sounds like we think there is a viable competitor out there for human general intelligence in the AI world. There isn’t.”
What we tested: There is currently no AI system that is a viable competitor to human general intelligence.
Assessment: UnresolvedSaid
“We know how to build machines that do math (augmentation); we don’t know how to build machines that can read tax codes (replacement).”
What we tested: We do not currently know how to build machines that can read and understand tax codes well enough to replace accountants.
Assessment: UnresolvedSaid
“Well, no. Augmentation is way easier, because you don’t have to solve the whole job.”
What we tested: Using AI to augment workers is much easier than replacing them, because augmentation does not require solving the whole job.
Assessment: SupportedSaid
“having a machine look at a radiology scan is very different from replacing the entire job of a radiologist”
What we tested: Having AI read radiology scans is very different from replacing the entire job of a radiologist.
What happened: The society's contemporary statement describes patient advocacy, safety, treatment and communication responsibilities alongside diagnosis. These substantive duties support the distinction between interpreting images and replacing the whole clinical role. (European Society of Radiology; Insights into Imaging, )
Assessment: UnresolvedSaid
“I hereby bet publicly that AI will not fully displace radiologists (as opposed to merely augmenting them) before the year 2033.”
What we tested: AI will not fully displace radiologists, as opposed to augmenting them, before 2033.
Assessment: UnresolvedSaid
“There is no evidence whatsoever that what has gotten us, sort of, into the game of reading scans is going to bring us all the way into the promised land of a radiologist in an app any time soon.”
What we tested: There is no evidence that current scan-reading AI techniques will produce a full radiologist replacement any time soon.
Assessment: SupportedSaid
“So if you have a correlation of R = .5, you actually “explain” (really, just “predict”) only 25% of the variance—which means fully three-quarters (not half) of the variability remains unexplained.”
What we tested: The study's R = 0.5 correlation predicts only 25% of the variance, leaving three-quarters unexplained, not half as the paper implied.
What happened: The paragraph ties this statement to R = 0.50. Squaring that coefficient yields 25% accounted for and 75% remaining. (Scientific Reports, )
Assessment: SupportedSaid
“It’s way too much charity to deep learning to claim that there is any kind of direct link.”
What we tested: The evidence does not support claiming any direct link between deep learning models' inner workings and the human brain.
What happened: The author's account describes mapping, correlations and effects of changing model depth and attention. Inferring that these do not establish identical biological mechanisms is the evaluator's assessment, not an explicit concession in this quotation. (Charlotte Caucheteux, reproduced by Thread Reader, )
Assessment: UnresolvedSaid
“We can’t really build humanoid domestic robots without general intelligence, and we don’t yet have any serious candidates.”
What we tested: Humanoid domestic robots require general intelligence, and there are no serious candidates for such intelligence yet.
Held from this post (4)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: A radiologist's job includes not just reading images, which convnets are suited to, but also, in some cases, reasoning about a patient's history and reading unstructured text, two problems for which convnets are less suited.
- The record does not meet the current rules. Our reading, not assessed: Convnets cannot solve pretty much everything, contrary to Yann LeCun's claim.
- The record does not meet the current rules. Our reading, not assessed: AI's ability to solve some aspects of radiology, such as scanning images, does not by any stretch of the imagination mean it can solve all aspects, such as reading clinical notes.