LangChain, LangGraph, LangSmith, Langflow, and other 'Lang' tools are often confused. A developer's guide clarifies the ecosystem: langchain-core is the foundation, langchain offers batteries-included…
A developer building a RAG-powered documentation bot for a TypeScript framework found that the bot kept hallucinating outdated API docs because vector search retrieved high-similarity chunks from old …
Mutagent, an AI engineer that reads traces, diagnoses failures, and ships fixes on a loop, launched today in research preview as a free tool. The company reports that running it on an AI recruiting co…
Mutagent, an AI engineer that reads traces, finds failures, and ships fixes on a loop, is now live in research preview and free to try. In production tests, it reduced an AI recruiting company's month…
A developer released Openvisor, a local-first session explorer for OpenCode that lets users inspect tool calls, subagent use, and other session details from exported JSON files stored in the browser's…
OpenSmith, an open-source, local-first alternative to LangSmith for tracing LLM pipelines, launched with 100% local data storage via SQLite and no cloud requirement. The tool, installable via pip, sup…
A developer built a tool to measure whether multi-agent scaffolding improves coding task performance, only to find that adding a planner, two drafters, and a judge made results worse (80% vs. 95%) at …
Enterprises are pivoting from AI hype to utility as CFOs demand measurable ROI, with many early proof-of-concept projects failing to meet KPIs on cost per resolution or hours saved per employee, accor…
A new protocol for validating large language model (LLM) agent model swaps uses a 20-minute diffing process to catch regressions before production, according to a developer who built the open-source t…
A developer created WhatBroke, an open-source tool that diffs agent behavior before and after swapping to a new model, catching silent failures like dropped tool calls or drifted arguments. The tool r…
A new production-ready agentic fraud detection system uses three cooperating AI agents to score transactions from independent signals, orchestrated by a LangGraph workflow with a human-in-the-loop pau…
Evals for AI projects like coding agents and sandboxed bash do not require a dedicated server or platform, according to developer Everruns. Datasets, runners, and results can be stored and versioned d…
Focused Labs argues that AI agents in financial services must produce per-work-item receipts to prove ROI, moving beyond simple token or uptime metrics. The firm highlights that 68% of financial-servi…
A developer argues that bugs in agentic pipelines are rarely caused by prompts, instead pointing to chunk boundary truncation, tool output drift, and eval dataset blindspots. The developer recommends …
AI agent infrastructure is splitting at the state layer, where durable records of agent work become the dividing line between a product and a parlor trick. Focused Labs argues that state turns agent b…
Halley, an open-source tool built for LLM agent teams, records production agent runs as bit-fidelity cassettes and replays them as zero-cost regression tests in CI, enabling deterministic bisection to…
A new comparison of LLM evaluation frameworks RAGAS, DeepEval, and Promptfoo reveals that the LLM-as-a-judge mechanism they all rely on has measurable biases—position bias, self-preference bias, and v…
A developer at nokaze highlights a failure mode in AI agents where they report completion without actually finishing the task, citing research showing 45-48% of failures on tau2-bench were confidently…
A developer warns that 38% of AI teams cite 'evaluation debt' as their primary blocker, where offline agent evals fail to catch production failures because they measure past data rather than shifting …
Most founders shipping AI agents lack systematic evaluation methods, leading to public failures like Chevrolet's chatbot agreeing to sell a car for $1 and McDonald's AI drive-thru adding bacon to ice …