When 45 yuan is the right number but the wrong answer: four Chinese evidence-reading examples

A maintainer of the 估号参谋 valuation tool for the PC game 梦幻西游 (Fantasy Westward Journey) published four JSONL records on Hugging Face that test whether a model matches a number or matches the claim behind it, using a public historical report listing three comparable pairs of shoes at 35, 45 and 50 yuan. The records show that the median of 45 yuan is an unsupported answer to "What did the buyer pay for those shoes?" because all three observations are asking prices, and that three actual ring sales lacking the target ring's seal-accuracy stat still do not establish a price for the target. The author states the four records describe one case and are meant for hand inspection of answers, not for ranking models or estimating today's prices.

I help maintain 估号参谋, a valuation tool for the PC game 梦幻西游 Fantasy Westward Journey . We made a small Chinese reading exercise from one public historical report. It exposed a distinction I’d want to check in a retrieval-based QA system: matching the number and matching the claim are two different tests. The report lists three comparable pairs of shoes at 35, 45 and 50 yuan. The median, 45, was used in the estimate. If the question is “What did the buyer pay for those shoes?”, 45 is an unsupported answer . All three observations are asking prices. A second example goes the other way: there are three actual sales for comparable rings, but none has the target ring’s seal-accuracy stat. Their median is easy to calculate and still doesn’t establish a price for the target. The four JSONL records on the Hub https://huggingface.co/datasets/zhangsan-ai-1027/mhxy-valuation-evidence-qa include the Chinese question, reference answer, evidence type, and link to the public source. All four describe one case. They’re examples for inspecting answers by hand; they don’t support ranking models or estimating today’s prices. A simple exercise is to give a model the relevant excerpt and question while holding back the reference answer. Check whether it 1 identifies whose item the price belongs to, 2 distinguishes a listing from a sale and an actual payment, and 3 leaves the amount unknown when the excerpt doesn’t establish it. For people annotating similar data, do you score the evidence type separately from the amount?