EvalSift – 100 hand-labeled LLM agent traces EvalSift Lite, a free MIT-licensed Python tool for error analysis on LLM agent traces, ships with a hand-labeled case study of 100 traces from a tool-using support agent in which one labeler marked 88 pass and 12 fail. The 12 failures break down as 7 cases of asking for details instead of calling the tool, 2 misapplied policy rules at the "over $75" edge, 1 mishandled ambiguous product name, 1 date arithmetic error off by one, and 1 out-of-scope answer. The tool requires Python 3.10+, has no dependencies or API key, validates a traces.jsonl file, grades each trace with deterministic code checks, and writes Markdown/HTML reports; its demo grades 8 traces from a fake support agent with 4 planted failures and catches all 4. Find out how your LLM agent actually fails before you write a single eval. EvalSift Lite is a free, MIT-licensed Python tool 3.10+, no dependencies, no API key for error analysis on LLM agent traces: it validates a traces.jsonl file, grades each trace with deterministic code checks, and writes a Markdown/HTML report of what failed. It comes with a case study: 100 traces from a tool-using support agent, labeled by hand. Get the full Solo kit 100 labeled traces + data : https://merletownley.gumroad.com/l/fjppx https://merletownley.gumroad.com/l/fjppx git clone