LAI #140: Our AI Coding Playbook

Louis-François Bouchard, Co-founder & Head of Community at Towards AI, shared the engineering team's playbook for shipping reliable code with AI coding agents, recommending that important stochastic LLM tests be run five times to catch intermittent failures that a single successful run can hide. He also announced a 90-minute live workshop with Packt on structuring an AI engineering setup around coding agents like Claude Code and Codex, and highlighted SkillNet, an open-source adaptive learning system built by community member Jose_sonder.

Good morning, AI enthusiasts There’s a big difference between getting an AI agent to write code and getting it to produce code you can actually ship. This week, I’m sharing the workflows we use at Towards AI to close that gap, from context and debugging to testing and review. I also have a simple rule for testing stochastic LLM systems that catches failures a single successful run can hide. And if you use Claude Code or Codex regularly, I’m opening up my own AI engineering setup in a live workshop with Packt, including how I work across agents, manage usage limits, and keep skills and knowledge synced. Plus, this week’s reads will help you: Let’s dive in This week, in What’s AI, I am sharing what the Towards AI engineering team uses to ship reliable code with AI agents, including three AI coding workflows, context rules, debugging habits, tests, and review guardrails. There is still a huge gap between the people who ship working products with agents and the people who ship working demos. This article shares everything you need to close that gap. By the end, you will have the best practices for your own repo, whether you are a developer learning to direct agents or a beginner upgrading from random vibe coding to something that ships. Read the full article here https://www.louisbouchard.ai/ai-coding-workflows-that-ship/ , or, if you prefer, watch the walkthrough on YouTube https://www.youtube.com/watch?v=9SCkcVKPEy0 . A student in our 10-Hour LLM Fundamentals Video Course https://towardsai.com/academy/llm-primer/?utm source=newsletter&utm medium=email&utm id=AItip asked us a useful question: How many times should you run an important test before trusting the result? For stochastic LLM tests, our practical baseline is five runs. Suppose a difficult test passes 90% of the time. If you run it once and it passes, you might conclude that everything is working. Run the same test five times, though, and there is about a 41% chance you will see at least one failure. For important tests, run the same case five times and record two things: Keep the prompt, model, temperature, and other randomness settings, as well as the source context, fixed. Otherwise, you are changing the test while trying to measure its consistency. Five runs is not a statistical guarantee. It is a simple way to catch intermittent failures that a single successful run can easily hide. — Louis-François Bouchard, Towards AI Co-founder & Head of Community In other exciting news, I’m doing a 90-minute live workshop with Packt https://www.eventbrite.co.uk/e/build-your-personal-ai-engineering-system-registration-1998640855601?aff=oddtdtcreator&keep tld=true on how I’ve structured my own AI engineering setup around coding agents like Claude Code and Codex. If you’re already using agents in your workflow, this should be especially useful. You’ll see how I: Who it’s for: AI engineers and developers already using coding agents who want a more structured system around them. Jose sonder https://discord.com/channels/702624558536065165/983037843532308500/1541583090646777866 has built SkillNet, an open-source adaptive learning system. Users can share a topic or existing material, and it builds a structured course that presents the same knowledge differently for each learner. It can run as a shared space for an organization or class, or as an individual learning workspace. In a dynamic course, the shared knowledge and objectives remain stable, while the explanation, activities, support, and interface adapt based on the learner’s declared preferences, role, level, and progress. Check it out on GitHub https://github.com/ANFAIA/SkillNet and support a fellow community member. If you have any questions or feedback, share them in the thread https://discord.com/channels/702624558536065165/983037843532308500/1541583090646777866 . Most of you are still on free or basic plans, which makes me wonder how coding agents actually fit into your work today. Are they already handling a meaningful share of your coding and automation, or are you still bringing them in for specific tasks where they are particularly useful? I’m also curious about the people at either end. If you’re paying for 5x or more, what makes the extra capacity worthwhile? Is it simply that you use agents much more often, or is staying within a familiar Claude Code/Codex workflow valuable enough that you would rather pay than move work elsewhere? And the 8% running locally might be the group I’m most curious about: what are you running, on what hardware, and which parts of your workflow have you been able to move local? There are probably some very different AI engineering setups hiding behind these percentages. Share yours in the thread, especially what you use, what you pay for, and what still runs locally https://discord.com/channels/702624558536065165/833660976196354079/1541469511147061388 . Meme shared by bin4ry d3struct0r https://discord.com/channels/702624558536065165/830572933197201459/1542229068894969866 How The Fast Fourier Transform Actually Works https://pub.towardsai.net/fourier-transform-signal-processing-beginners-guide-44ba53b18ed5?sk=bf450dfe30c49412abc91cd3f93653f8 by Kamrun Nahar https://iknahar.medium.com/?source=post page---byline--44ba53b18ed5--------------------------------------- If you have used a Fast Fourier Transform FFT without really understanding what happens underneath, this article builds the intuition from the ground up. It starts with sine waves, implements the Discrete Fourier Transform in Python, and checks the result against NumPy before explaining how the FFT makes the same computation practical at scale. You’ll also see how sampling rate, aliasing, spectral leakage, and windowing affect the frequencies you recover from real signals, then apply the ideas to spectrograms, compression, MRI, and audio forensics. 1. Azure Storage for AI Inference: Prompt Caching, KV Offload, and Faster Model Loading https://pub.towardsai.net/azure-storage-for-ai-inference-prompt-caching-kv-offload-and-faster-model-loading-dbb7350ac414?sk=e0f4cd43aec9d876c20fd2675ab27885 by Dave R https://blog.azinsider.net/?source=post page---byline--dbb7350ac414--------------------------------------- Inference performance depends on more than GPU compute. This article shows how storage can reduce the amount of work your GPUs repeat. It explains Microsoft Foundry’s three levels of prompt caching, including Azure Context Cache for sharing cached prefixes across nodes, then shows how NIXL and LMCache move KV caches beyond GPU memory. You’ll also learn how Run:ai Model Streamer and AKS caching reduce model-loading time, giving you a practical map of where storage can improve latency, GPU utilization, and inference cost. 2. Your KV Cache Is Bigger Than Your Model https://pub.towardsai.net/your-kv-cache-is-bigger-than-your-model-59cdf28c1e6b?sharedUserId=tai-tech by Satsawat Natakarnkitkul https://medium.com/@net satsawat?source=post page---byline--59cdf28c1e6b--------------------------------------- Long context can exhaust GPU memory even when the model itself fits comfortably. This article shows you exactly why by calculating KV-cache memory from a model’s architecture, sequence length, precision, and batch size. Using gpt-oss-120b on two H100s shows that capacity drops from 504 concurrent conversations at 4K tokens to just 15 at 128K tokens. It then works through six ways to recover capacity, including grouped-query attention, sliding windows, prefix caching, paging, and FP8 quantization. 3. Watermarking Text Generation Efficiently https://pub.towardsai.net/watermarking-text-generation-efficiently-8edbffb1cf65?sk=ae31cd0eb22d17fee4be2e050309ff4c by Enzo Lombardi https://enzolombardi.net/?source=post page---byline--8edbffb1cf65--------------------------------------- Text watermarking can work without inserting hidden characters or maintaining a list of special words. This article explains how watermarking can modify token probabilities during generation, including the green-list approach and the distortion-free method used in Google’s SynthID-Text. It then derives how a simple z-score can detect the resulting statistical pattern and tests both methods inside a real inference engine. You’ll also see where the approach breaks down: changes in paraphrasing, translation, and tokenization can weaken or remove the signal. 4. Beyond RAG: Implementing Agentic Memory Architectures for Long-Horizon AI Systems https://pub.towardsai.net/beyond-rag-implementing-agentic-memory-architectures-for-long-horizon-ai-systems-3e72b8b0ec84?sk=1959caac23b38d4847bf98e33dc9fae8 by Sourav Mukherjee https://medium.com/@sourav15?source=post page---byline--3e72b8b0ec84--------------------------------------- RAG can retrieve what an agent needs in the moment, but it does not automatically help the agent learn from past events. This article builds a memory architecture that separates episodic memory what happened , semantic memory what the agent learned , and procedural memory how it should behave . It then adds consolidation to turn past interactions into reusable knowledge and reflection to promote successful lessons into future behavior. Comparisons with Mem0, Letta, and LangGraph help you see how these ideas translate into existing memory frameworks and what you need beyond retrieval for agents that operate across many sessions. If you are interested in publishing with Towards AI, check our guidelines and sign up https://contribute.towardsai.net/ . We will publish your work to our network if it meets our editorial policies and standards. LAI 140: Our AI Coding Playbook https://pub.towardsai.net/lai-140-our-ai-coding-playbook-d97f2bed5f35 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.