ls /news · home › news
grep -r --recent /news | head -20

News

161676 articles page 80 of 8084 0 sources 30 min sync cycle

// latest articles 161676 indexed

00:00
2026-11-18
superml.org
ai-agents · 8m read · neu

Capstone: Build an Agent Evaluation Framework

A capstone lesson in an agent evaluation course instructs engineers to assemble a complete, layered evaluation framework for a tool-using agent and ship it in a repository with CI. The deliverable requires at least 120 s…

00:00
2026-11-18
superml.org
ai-agents · 10m read · neu

Evals in CI/CD and Regression Gates

A lesson on evaluating agentic AI systems recommends wiring eval suites into CI/CD so that every change capable of altering agent behavior — agent code, system prompts, model versions, tool definitions, retrieval content…

00:00
2026-11-18
superml.org
ai-agents · 9m read · neu

Eval-Driven Agent Architecture

A lesson on eval-driven agent architecture argues that agents should be built around four explicit seams — model, tool, state and trace — so that a complete structured record of every run is produced automatically. The l…

00:00
2026-11-18
superml.org
ai-agents · 8m read · neu

Designing the Evaluation Framework

A tutorial on building an agentic AI evaluation framework lays out four evaluation layers — component, step, trajectory, and outcome — and five core abstractions — Case, Run, Grader, Score, and Report — in roughly 200 li…

00:00
2026-11-18
superml.org
ai-agents · 8m read · neu

Golden Datasets and Scenario Suites

A lesson on building evaluation datasets for AI agents argues that the golden dataset, not the grader, is the most important artifact in an evaluation framework because it defines what counts as correct behavior. The les…

00:00
2026-11-18
superml.org
ai-agents · 9m read · neu

LLM-as-Judge Done Right

A tutorial on LLM-as-judge evaluation recommends splitting grading into narrow single-criterion judges with binary or low-cardinality rubrics rather than one broad 1–10 quality score, and calibrating each judge against h…

00:00
2026-11-18
superml.org
ai-agents · 8m read · neu

Multi-Turn, Memory, and Multi-Agent Evaluation

A tutorial on multi-turn, memory, and multi-agent evaluation argues that single-turn LLM evals miss conversational failures such as re-asking for information already given, forgetting user constraints, over-asking and un…

00:00
2026-11-18
superml.org
ai-agents · 8m read · neu

Online Evaluation and Production Monitoring

A lesson on online evaluation and production monitoring for agentic AI systems argues that offline evaluation only measures past, curated behavior, while online evaluation must judge live production runs that carry no ex…

00:00
2026-11-18
superml.org
ai-agents · 9m read · neu

Reliability, Cost, and Latency

A lesson on agent reliability evaluation argues that teams should report pass^k — the probability that all k attempts succeed — rather than pass@k for user-facing agents, citing τ-bench as the benchmark that popularized …

00:00
2026-11-18
superml.org
ai-safety · 8m read · neu

Safety, Red-Teaming, and Adversarial Evaluation

A new lesson on safety, red-teaming and adversarial evaluation argues that tool-using AI agents require eval suites testing what the agent does under attack, not just lists of jailbreak prompts, because a tricked agent c…

00:00
2026-11-18
superml.org
ai-agents · 8m read · neu

Why Agent Evaluation Is Different

Agent evaluation requires statistical, multi-run testing because agents are loops rather than functions, according to a tutorial on agentic AI evaluation that outlines five properties making agents harder to evaluate tha…

← prev page 80 / 8084 next →
LIVE [news] indexed:161676 page:80/8084 en · ua 2026-05-20 · —