WuYuEval: A Multi-Level Benchmark for Large Language Models in Solid Waste Management

Researchers introduced WuYuEval, a multi-level benchmark for evaluating large language models in solid waste management, containing 4,590 multiple-choice questions and 247 scenario-based expert questions. Across 33 LLMs, the leading model reached 94.64% accuracy on the Foundation Module, but average accuracy dropped from 84.14% on easy questions to 42.50% on hard ones, with reasoning modes improving performance only when anchored to engineering constraints.

arXiv:2608.07529v1 Announce Type: new Abstract: Large language models LLMs are increasingly used as technical assistants, but their competence in solid waste management SWM remains difficult to assess because existing benchmarks emphasize general knowledge rather than professional decisions under engineering, environmental, and policy constraints. We introduce WuYuEval, a multi-level benchmark for evaluating LLMs in SWM across foundational knowledge, domain reasoning, and expert decision-making. After quality auditing, WuYuEval contains a Foundation Module with 4,590 closed-ended multiple-choice questions across six task types and eight domain categories, together with an Expert Module with 247 scenario-based open-ended questions involving multi-objective optimization, constraint trade-offs, and system design. For expert tasks, we combine anchor-calibrated LLM-as-a-Judge scoring with Elo-based pairwise comparison. Across 33 LLMs, performance varied widely. The leading model reached 94.64\% accuracy on the Foundation Module, but average accuracy still fell from 84.14\% on easy questions to 42.50\% on hard questions, with lower performance concentrated in calculation, experimental design, urban planning, and open-ended expert tasks. Reasoning-oriented Thinking modes improve most matched model pairs after auditing, but the gains depend on baseline capability and are not uniformly positive. These results suggest that visible deliberation helps only when it remains anchored to units, assumptions, and engineering constraints; otherwise, it may drift from decisive answer boundaries. WuYuEval therefore provides both an evaluation resource and an empirical basis for developing SWM-oriented foundation models with professional reasoning chains and explicit constraint control.