Good morning, AI enthusiasts!
Coding agents can now take on enough work that the question is no longer just how much faster they make us. It’s how closely we still need to watch them.
This week, I look at where vibe coding works well, where it starts to get risky, and what I’ve learned about using coding agents without handing over the engineering judgment with the code. There’s also a small observability detail that can completely distort your reliability and cost numbers if you get it wrong.
Then, a few reads worth your time:
We also finally share Towards AI Mentorship here, plus a new community-built coding copilot that reports cutting codebase context by 70–80%.
Let’s get into it!
This week, in What’s AI, I dive into something that has become an integral part of software development: vibe coding. Vibe coding with AI agents can speed up software work, but what I want to talk about is how it is a game changer only IF you use it correctly. To use it right, you need good context, tests, reviews, and task boundaries. The risky version of vibe coding is shipping whatever the agent produced because it looked like it worked. Read how to do it right or watch the video version on YouTube.
A retry is not a new user request. Your traces should reflect that.
In the Opik observability lesson from our Agent Engineering course, we trace model calls and tool calls across an agent run. One issue that comes up quickly is how retries should be recorded.
If every retry is counted as a separate request, a single user request can appear several times in your dashboard. That inflates request volume and makes it harder to see how many attempts the agent actually needed to succeed.
Use the same request ID across every retry, and add an attempt number for each one. Keep separate trace and span IDs for the individual operations.
This lets you measure both the number of user requests and the number of attempts required to complete them.
That distinction is important to note for cost and reliability. A request that succeeds after three attempts may look successful in the dashboard, while using far more time and tokens than a request that succeeds on the first try.
— Louis-François Bouchard, Towards AI Co-founder & Head of Community
A couple of weeks ago, we opened something new at Towards AI that I haven’t had a chance to share here yet.
One thing we kept hearing from students was that learning the material wasn’t always where they got stuck. The harder questions came afterward: Is this architecture actually a good idea? Why is my agent failing with real users? Is this project strong enough for my portfolio? Why isn’t my resume getting through?
Those questions are difficult to solve with another lesson or another chatbot response because the answer depends on your specific work.
So we now have Towards AI Mentorship, where you can bring those questions directly to our team of 15 senior AI engineers. That includes async technical and career help, live sessions twice a week, resume and project reviews, monthly production blueprints from our deployment work, and workshops with engineers working in the field.
It’s $99/month, and you can use it whether you’re trying to land an AI role or already building AI systems and want experienced engineers to sanity-check the decisions you’re making.
Learn more about Towards AI Mentorship
Ang_0007 built PariPari, a repo-aware AI copilot that uses Paritok context compression to explore massive codebases, fix bugs, and generate PRs without blowing up LLM token limits or budgets. It is built with Python and FastAPI to manage the agent loop, tool execution, and GitHub API interactions, and uses the Groq API. He reports achieving an average of 70–80% token reduction on codebase file reads using Paritok compression. Check it out and support a fellow community member. If you have any questions or feedback, share them in the thread.
The results barely moved as more people voted: 57% still run long coding-agent tasks on their main machine. What I find more interesting is how fragmented the other 43% are. There doesn’t seem to be a clear second choice yet. Some people rely on provider-hosted environments, while others have moved to dedicated machines or their own servers.
That makes me curious about what these setups actually look like in practice.
For those running agents somewhere other than your main machine: what does your setup look like? What are you running, where does the code live, how do you connect to the agent, and do you typically have one task running or several at once? I’d love to hear what has actually worked for you. Meme shared by supastishn
A Field Guide to Agentic Eval Frameworks: Langfuse, LangSmith, and What to Measure by MongoDB
This article examines why traditional unit tests miss trajectory-level failures in AI agents, using a research agent that hallucinated a published report, misused a search tool, and looped through failed warehouse queries. It defines six evaluation dimensions: task success, trajectory quality, tool correctness, safety, factual accuracy, and cost, then outlines rule-based checks, LLM-as-judge grading, and human review across component, trajectory, and end-to-end testing.
- Microsoft’s Four IQs: How Foundry IQ, Fabric IQ, Work IQ, and Web IQ Ground Enterprise Agents by Dave R
This article breaks down Microsoft IQ, the context layer that grounds AI agents in four kinds of enterprise knowledge: unstructured documents, structured business data, human work signals from Microsoft 365, and fresh information from the web. It covers what each of the four engines (Foundry IQ, Fabric IQ, Work IQ, and Web IQ) does, how they fit together inside a single agent architecture, and where Foundry IQ unifies the others. It also walks through a worked refund processing agent so you can see how the pieces cooperate, why your agent instructions still matter after the IQs remove most of the plumbing, and how Agent 365 gives an agent its own identity and security boundary.
A scheduling flaw in Claude Code’s ScheduleWakeup mechanism re-fires one-shot slash commands instead of resuming d tasks, silently doubling API costs on database writes, pull requests, and expensive searches. This article shows how to choose between loops, skills, subagents, and workflows without burning budget.
Multi-agent LLM systems break provider billing dashboards, which report spend by key or model but never by agent, workflow, or trace. This piece details a metering layer that captures token usage at each call, enriches it with runtime context, and prices it against a versioned rate card stored in a separate collection. Aggregation pipelines then slice costs by agent, model, or outcome, feeding Atlas Charts dashboards.
This article explains how Rayfin works, from the TypeScript you write to the services that run in Microsoft Fabric. Rayfin is an open source SDK and CLI that lets you define an application backend, including its database, access policies, APIs, and server-side logic, entirely in code, then deploy it to Fabric with a single command. It walks through the programming model, the decorators that turn classes into tables, the CLI workflow, how connectors reach existing data, and how the whole application lands as a governed Fabric artifact.
If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards. LAI #138: The Agent Reality Check was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.