How come smart assistants have virtually no ability to converse, despite all the spectacular progress with large language models?
We recorded 10 claims from this post. 6 were checked against evidence. 4 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (10)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: ContradictedSaid
“It's worth asking since (a) Amazon is apparently greatly reducing Alexa's staff, (b) hasn't added significant features in years, and (c) seems clumsy and limited compared to GPT-3.”
What we tested: Alexa has not added significant features in years and seems clumsy and limited compared to GPT-3.
What happened: The dated launch account describes multiparty interaction and speech directed at Alexa without repeatedly saying the wake word. This is a substantive interaction feature within the claimed period, although it does not establish GPT-3-like open conversation. (Amazon Science, )
Assessment: Can’t be checkedSaid
“Amazon probably *is* using large language models for product search.”
What we tested: Amazon is probably using large language models for product search.
Assessment: Can’t be checkedSaid
“LLMs are inherently unreliable. If Alexa were to make frequent errors, people would stop using it.”
What we tested: Marcus suspects that a contributing reason Alexa does not converse is that LLMs are inherently unreliable, and if Alexa made frequent errors, people would stop using it.
Assessment: SupportedSaid
“LLMs are unruly beasts; nobody knows how to make them refrain 100% of time from insulting users, giving bad advice, or just plain making stuff up.”
What we tested: LLMs are unruly, and nobody knows how to make them refrain 100% of the time from insulting users, giving bad advice, or making stuff up.
What happened: OpenAI described reliable refusal of unsafe instructions as unsolved despite improvements from human-feedback training. Its limitations section also reports remaining toxic outputs and fabricated facts. (OpenAI, )
Assessment: Can’t be checkedSaid
“Any one of these scenarios of LLMs gone awry (bad advice, insults, lies etc) could hurt the Amazon brand, open up litigation, etc..”
What we tested: Any one of the scenarios of LLMs going awry (bad advice, insults, lies, etc.) could hurt the Amazon brand and open up litigation.
Assessment: UnresolvedSaid
“if Alexa could converse freely, user expectations would go through the roof, and mostly be unmeetable.”
What we tested: If Alexa could converse freely, user expectations would rise to a level that mostly could not be met.
Assessment: Can’t be checkedSaid
“they are not (and may never be) great tech for reliably inferring user intent from what users say.”
What we tested: LLMs may never become great technology for reliably inferring user intent.
Assessment: SupportedSaid
“Bottom line: From the outset Large Language Models like GPT-3 have great at generating surrealist prose, and they can beat a lot of benchmarks, but they are not (and may never be) great tech for reliably inferring user intent from what users say.”
What we tested: LLMs like GPT-3 are not great technology for reliably inferring user intent from what users say.
What happened: The study tests implied meaning rather than fluent text production and finds substantial shortcomings even in adapted models. It supports the ordinary reading that dependable contextual understanding remained a weakness, while acknowledging meaningful improvements. (Ruis et al., arXiv, )
Assessment: UnresolvedSaid
“See also my essay on Google’s Palm-SayCan, for what might happen if this tech were embedded in robots, which would be even more risky.”
What we tested: Embedding LLM technology in robots would be even more risky than using it in home assistants.
Assessment: UnresolvedSaid
“Turning LLMs into a product that controls your home and talks to you in a way that would be reliable enough to use at scale in millions of homes is still a long, long way away.”
What we tested: A reliable LLM-based conversational home-control product usable at scale in millions of homes is still a long, long way away.
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: Marcus does not think scaling is the real reason Amazon has not built conversational ability into Alexa.
- The two reviews reached different assessments. Our reading, not assessed: A sentence produced by an LLM cannot directly be used to control things unless another system is built to parse sentences into actions, and nobody knows how to build that reliably.