Good morning, AI enthusiasts!
A lot of the frustrating bugs in AI systems now happen after the model has already done its part.
The policy agent gets the right answer, but drops an important condition when it hands the task off. The batch job finishes, but your app never hears about it. Your latency dashboard looks fine, but the user is still waiting too long for the first token. Or the coding agent completes the command and leaves the credential sitting in its transcript.
This week’s issue has a lot of useful examples of how to deal with those problems. You’ll learn:
Let’s get into it!
This week in What’s AI, I’m sharing the setup I now use to switch between Claude Code and Codex without rebuilding context every time.
The main change was moving my notes, skills, scripts, meeting recaps, and recurring instructions into an Obsidian vault that both agents can read. An index tells them what to load for each task, so I can give an agent the context it needs without passing my entire archive into every prompt. I also keep feedback with each skill, so repeated corrections can become part of the default workflow.
I still use Codex, but I often let Claude Code plan the work and call Codex through MCP to execute it. I also move non-urgent jobs, such as daily news recaps, into scheduled runs so they do not compete with my daytime usage. The idea is to keep useful context and skills outside any one model or coding tool, so changing agents doesn’t mean starting over.
The full article goes through the vault structure, the role of skills and feedback files, how I split planning from execution, and how the setup follows me across devices. Read it here or watch the video on YouTube.
When developers first give an agent search access, relevance is often the main filter: find pages related to the query, then let the model read them.
That becomes expensive as the system scales. More searches bring in more pages, and relevant pages often overlap. You waste context, add latency, and spend money processing the same information multiple times.
We ran into this while scaling Nova, the research agent we built in Agent Engineering. Its searches kept surfacing shady blogs, sales pitches, low-quality pages, and several sources making essentially the same point.
So we stopped ranking sources on relevance alone. Nova now checks three things:
Sources that pass the first filters stay in the research file as short summaries. Nova then chooses only five to read in full, partly based on whether they add something missing from the material already collected.
If your agent already ranks search results by relevance, add novelty before the full page. Compare each source with what the agent already has. If it mostly repeats the same information, keep the summary or skip it. — Louis-François Bouchard, Towards AI Co-founder & Head of Community
My AI Engineer talk on why AI writing still reads like slop is now live on YouTube!
I walk through ToneBench, where we tested 78 models across 127 configurations using 6,350 generated scripts and 25,400 judge assessments. I cover how we measure tone, substance, structure, hooks, slop, and consistency, and what the results show about model choice, reasoning effort, and cost.
I also get into one of the harder parts of benchmarking writing: the judges themselves have measurable preferences, even when model identities are hidden.
Panelreader. built a neural network from scratch in Python and NumPy to understand what frameworks like PyTorch and TensorFlow handle behind the scenes. The project implements a 2 → 3 → 1 XOR network with forward propagation, backpropagation, and gradient descent written manually. After 5,000 training epochs, the network correctly predicted all four XOR cases with a final MSE of 0.004715. The repo also includes mathematical explanations, training visualizations, and 12 editable diagrams that walk through how the network works. Check it out on GitHub and support a fellow community member. If you have feedback on the code or documentation, share it in the thread!
The Learn AI Together Discord community is flooding with collaboration opportunities. If you are excited to dive into applied AI, want a study partner, or even want to find a partner for your passion project, join the collaboration channel! Keep an eye on this section, too — we share cool opportunities every week!
-
Hendrickop is looking for an AI/ML learning partner. He is a beginner but wants to learn quickly and work on projects. If you are following the same path, connect with him in the thread!
-
Martocierny is putting together a team to build a 2D multiplayer sandbox in the browser and is looking for coders, artists, testers, and even complete beginners. This is a hobby project, but if it sounds fun, reach out to him in the thread!
-
Gian_16992 is moving from backend orientation to AI engineering and is looking for people to discuss ideas, share resources, make projects, etc. If this sounds like a good next step for you, reach out to them in the thread!
Meme shared by bin4ry_d3struct0r
Dropout Finally Explained: Why Breaking Half Your Neural Networks Makes It Smarter by Simple ML
Neural networks can fit their training data closely but still perform much worse on new examples. Dropout addresses this by randomly removing units during training, so the network repeatedly trains different weight-sharing subnetworks instead of allowing the same neurons to depend on one another. This article works through the original dropout paper, explains co-adaptation and the weight-scaling rule, and connects dropout to regularization in linear models. It also reviews results across vision, speech, text, and genetics, along with practical guidance on keep probabilities, network size, max-norm constraints, and the additional training time dropout requires.
Traditional API monitoring misses several problems specific to LLM applications. A streaming request can have acceptable total latency while still making the user wait too long for the first token, and token usage adds a cost dimension that normal request metrics don’t track. This article shows how to measure time to first token, time per output token, token usage, and cost with Prometheus, separate high-volume metadata from full prompt logs, and connect requests across OpenTelemetry traces with a shared trace_id. It also covers redaction, sampling, and alerting so teams can trace a slow or expensive request back to the component that caused it.
A multi-agent system can return the correct final message even when one agent makes a serious mistake along the way. In the article’s example, a policy tool correctly returned that a damaged item had to be sent back before a refund, but the policy agent omitted that condition when handing the case to the orchestrator, so billing issued the refund immediately. The article shows how to evaluate each agent’s tool use, the information passed between agents, and the final outcome separately. It also runs cases multiple times to detect inconsistent behavior and uses step-level failure attribution to identify where the error entered the trajectory.
Polling a batch API creates a failure case when the batch finishes, but the process checking its status stops running. This article replaces polling with a TypeScript webhook flow designed for duplicate deliveries, temporary failures, and unavailable downstream services. It implements signature verification, atomic deduplication, idempotent side effects, exponential backoff, a dead-letter queue for failed events, and a circuit breaker for unhealthy endpoints. It also compares how providers handle batch completion and retries, and shows how to replay failed deliveries without processing the same job twice.
Coding agents often need credentials to run commands, but pasting a password or API key into chat leaves it in the model context and can also store it in local session transcripts. After examining 661 Claude Code transcripts, this article builds lsv, a Rust vault that lets an agent pass a secret into a command without providing a normal command for reading the value back. Secrets enter through the clipboard, appear as redacted references during use, and require Touch ID approval before access.
If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards. LAI #148: What Happens Between Prompt and Output? was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.