Getting GPT to work with external tools is harder than you think
We recorded 16 claims from this post. 11 were checked against evidence. 5 can’t be checked. 4 more are held. Read the post at its source.
Claims from this post (16)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
“GPT-4 and similar current language-based AI programs notoriously struggle with mathematical calculations.”
What we tested: GPT-4 and similar current language-based AI programs struggle with mathematical calculations.
What happened: Table 3 reports exact-answer accuracy of 18.84% for GPT-4 and 10.00% for ChatGPT on 9,592 arithmetic cases. These substantial error rates support the ordinary reading of calculation difficulty. (arXiv; Zhen Yang and colleagues, )
Assessment: Can’t be checkedSaid
“That seems like a sensible idea; indeed, it is a crude version of the kind of hybrid neuro-symbolic system that we have been advocating for many years.”
What we tested: Having GPT-4 use external tools such as the Code Interpreter and Wolfram Alpha plug-ins seems a sensible idea, and it is a crude version of the hybrid neuro-symbolic system the authors have advocated for many years.
Assessment: SupportedSaid
“Certainly, adding these plug-ins does improve things.”
What we tested: Adding the Code Interpreter and Wolfram Alpha plug-ins improves GPT-4's math performance.
What happened: On the common 120-problem subset, Figure 4a reports 48.33% success for ChatGPT with Code Interpreter, 45.0% with Wolfram Alpha, and 30.0% for vanilla GPT-4. The commercial products were not evaluated on the entire MATH test set. (arXiv; Qingyun Wu and colleagues, )
Assessment: SupportedSaid
“Second, GPT-4 doesn’t reliably understand how to use the tools.”
What we tested: GPT-4 does not reliably understand how to use the Code Interpreter and Wolfram Alpha plug-ins.
What happened: By completion of testing, the report documented repeated unsuccessful Wolfram queries, failure to request a usable numerical representation, and Code Interpreter bypasses that produced calculation errors. These distinguish tool-use failures from merely misunderstanding a science problem. (Ernest Davis and Scott Aaronson, hosted by New York University, )
Assessment: UnresolvedSaid
“First, neither GPT-4 nor the plug-ins have the commonsense world knowledge and reasonable abilities needed to reliably translate a word problem into a mathematical calculation.”
What we tested: Neither GPT-4 nor its plug-ins have the commonsense knowledge and reasoning needed to reliably translate word problems into calculations.
Assessment: SupportedSaid
“On some problems that capable students might well find hard, the AI did perfectly; on other problems that lot of mathematically-inclined college students people would find easy, the AI failed miserably.”
What we tested: In the authors' tests, GPT-4 with plug-ins did perfectly on some problems that capable students might well find hard, and failed miserably on other problems that many mathematically inclined college students would find easy.
What happened: The report identifies correct high-dimensional probability and finite-field linear algebra answers alongside elementary failures. Their mathematical prerequisites support the qualitative difficulty contrast, but the authors explicitly report no human-subject testing. (New York University; Ernest Davis and Scott Aaronson, )
Assessment: UnresolvedSaid
“In a few cases GPT-4 set up the calculations correctly and the plug-in returned the correct answer, but then GPT-4 either misunderstood or refused to accept that answer.”
What we tested: In a few test cases GPT-4 set up the calculations correctly and the plug-in returned the correct answer, but GPT-4 then either misunderstood or refused to accept that answer.
Assessment: SupportedSaid
“But, crucially, it often gets these wrong, particularly when they involve spatial reasoning beyond a cookie-cutter formula or reasoning about sequences of disparate events.”
What we tested: GPT-4 with either plug-in often gets wrong problems that a human solver would need significant thought to translate into math notation, particularly those involving spatial reasoning beyond a cookie-cutter formula or reasoning about sequences of disparate events.
What happened: Both systems failed the pendulum and chemical-reaction problems. The spatial eclipse subset contained six wrong answers out of eight for GPT4+WA and five out of eight for GPT4+CI. These are repeated failures in the described collection. (New York University; Ernest Davis and Scott Aaronson, )
Assessment: MixedSaid
“In many cases, a naive user might take the results, invariably presented as fact, as correct, when they are not.”
What we tested: GPT-4 with plug-ins invariably presents its results as fact, and in many cases its results are incorrect even though a naive user might take them as correct.
What held: The answer key for problem 3 is F, while the output confidently selects H. Problems 1 and 4–7 also contain incorrect declarative answers, supporting the recurring confident-error component. (New York University; Ernest Davis and Scott Aaronson, )
What did not: The life-in-the-universe calculation explicitly warns about uncertainty in its estimates and repeats that qualification after reporting percentages. This contradicts the absolute assertion that results invariably appear as unqualified facts. (New York University; Scott Aaronson and Ernest Davis, )
Assessment: SupportedSaid
“There is no question that the two plug-ins significantly extend GPT-4’s ability to solve math and science problems.”
What we tested: The Code Interpreter and Wolfram Alpha plug-ins significantly extend GPT-4's ability to solve math and science problems.
What happened: On the matched 120-problem subset, figure 4(a) reports 48.33% success for ChatGPT with Code Interpreter, 45.0% with Wolfram Alpha, and 30.0% for vanilla GPT-4. These are substantial practical gains for both plug-ins, although the comparison does not fully isolate differences between the web products and baseline configuration. (Qingyun Wu and colleagues, arXiv, )
Assessment: Can’t be checkedSaid
“Some of these problems may fixable. With enough resources OpenAI or Wolfram Alpha could fix some of the problems in the interfaces.”
What we tested: With enough resources, OpenAI or Wolfram Alpha could fix some of the interface problems.
Assessment: Can’t be checkedSaid
“All that said, much more fundamental problems remain and, until those problems are addressed, the ability of AI to apply mathematics to real world situation will remain limited.”
What we tested: Until fundamental problems are addressed, AI's ability to apply mathematics to real-world situations will remain limited.
Assessment: Can’t be checkedSaid
“A true understanding of elementary mathematics and its uses requires a deeper view.”
What we tested: True understanding of elementary math requires more than LLM token prediction or Wolfram-style symbol manipulation.
Assessment: SupportedSaid
“Finding a way to get LLMs to reliably use external tools is far from trivial.”
What we tested: Getting LLMs to reliably use external tools is far from trivial.
What happened: The July benchmark reports low completion rates for agents operating realistic websites with supporting tools, despite incorporating reasoning before acting. This supports the inference that connecting models to an actionable environment did not itself deliver dependable performance. (arXiv; Shuyan Zhou and colleagues, )
Assessment: UnresolvedSaid
“Thus far we have seen consistently unreliable results, from math to fact-checking.”
What we tested: So far, the authors have seen consistently unreliable results from LLMs using external tools, from math to fact-checking.
Assessment: Can’t be checkedSaid
“We don’t claim that this problem is unsolvable, but addressing it may require major innovation.”
What we tested: Getting LLMs to reliably use external tools may require major innovation, though it is not necessarily unsolvable.
Held from this post (4)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: In the tests, GPT-4 often made syntax errors calling Wolfram Alpha and sometimes could not correct them.
- The two reviews reached different assessments. Our reading, not assessed: Even with plug-ins, GPT-4 cannot reliably solve word problems involving simple math or simple science and math.
- The two reviews read the claim differently
- The two reviews read the claim differently