A model that summarizes a table adds its own sums

A developer's test found that Claude Sonnet computed its own figures when asked to summarize 24 small tables using only printed numbers, adding derived totals, ratios and differences on eight tables, while Claude Haiku added none. Every computed figure was arithmetically correct, but once embedded in prose a reader cannot distinguish it from a number the table actually contained.

We gave 24 small tables to Sonnet and Haiku and told both to use only printed figures. Sonnet still computed its own on eight tables, Haiku on none. Yes, and the stronger model did it more often. Our test gave Claude Sonnet and Claude Haiku 24 small tables, and the job each time was to summarize a table in plain prose, staying inside the numbers it prints. On eight of those tables Sonnet still produced a number or a comparison that was nowhere in the data, such as a difference, a ratio or a total. Haiku did that on none. Every computed figure was correct arithmetic, which is the problem: once it sits in a paragraph, a reader cannot tell it from a number the table itself carried. Read the full report on AISkills402: https://aiskills402.com/blog/llm-table-summary-adds-figures?utm source=devto&utm medium=social https://aiskills402.com/blog/llm-table-summary-adds-figures?utm source=devto&utm medium=social