Context engineering with Dex Horthy

Dex Horthy, CEO of HumanLayer, coined the term 'context engineering' and discussed on The Pragmatic Engineer podcast how LLM context limitations affect software development. He shared lessons from building AI agents, including that shipping unread code leads to disaster and that current coding models may degrade codebases over time.

Stream the latest episode Listen and watch now on YouTube, Spotify , and See the episode transcript at the top of this page, and timestamps for the episode at the bottom. Apple https://podcasts.apple.com/us/podcast/the-pragmatic-engineer/id1769051199 . Brought to You by • Antithesis — with Antithesis, you can use AI agents to work on critical systems without worrying about correctness. Going far beyond code review, you can run your complete system in a hostile environment, analyze its behavior, and reproduce every issue perfectly. Teams like Jane Street, Fly.io http://fly.io , and the etcd community use Antithesis to ship better code, faster. Learn more. https://antithesis.com/pragmatic • Buildkite — the CI platform trusted by OpenAI, Anthropic, Cursor, Meta, Uber, Ramp, Nvidia, Airbnb and many more. Buildkite was stress-tested at the largest scale inside companies solving some of the hardest engineering problems. It’s built to reliably manage whatever your coding agents throw at the build queue, today, next year and beyond. Learn more. https://buildkite.com/pragmatic • Sentry – application monitoring software built by developers, for developers. Sentry’s Seer AI agent https://sentry.io/lp/seer/utm source=pragmatic&utm medium=paid-community&utm campaign=ai-fy27q1-seerlaunch&utm content=recording-coreread-freetrial&code=pragmatic is one of their new, neat tools, as a way to debug errors faster. Same with Sentry’s MCP server. Check out Sentry http://sentry.io/pragmatic . In this episode Knowing how LLM contexts work and how to work around context limitations – aka “context engineering” – is becoming more important for software engineers working with LLMs. Let’s look into what works and what doesn’t, today. In this episode of The Pragmatic Engineer podcast, I sit down with the CEO and cofounder of HumanLayer, Dex Horthy, who coined the term “context engineering”. We discuss the ideas behind this context engineering, harness engineering, loop engineering, software factories, why his approach to AI-assisted software development has evolved, and how HumanLayer https://www.humanlayer.dev/ is helping engineering teams automate more of the software development lifecycle without sacrificing code quality. Key observations from Dex Here are 12 useful points Dex made in our conversation: 1. Dex talked with ~100 “real” AI Engineers and wrote the popular ‘ 12-Factor Agents - Principles for building reliable LLM applications ’ based on what he learned. Around August 2024, he started to build AI agents when the common approach was to use frameworks like LangChain and CrewAI, and also talked with around 100 AI engineers doing tangible work such as taking on $100K+ contracts to ship AI solutions within enterprises. They tried those frameworks and discarded them in favor of building their own pipelines. Dex shared the learnings from the conversations in his hit book, 12-Factor Agents https://github.com/humanlayer/12-factor-agents . It’s a great read 2. Lesson learned: Shipping unread code spells disaster within months. Dex experimented with having the model write the code and humans not reviewing anything in July 2025. Four months later, they shut things down and threw the whole system out. Production broke, and no matter how much the team prompted Opus 4.1, the model could not find the root cause. It took days of wading through spaghetti code to discover the primary key wrongly routed through the complete codebase. Once fixed, it took three weeks to re-onboard to a codebase no human had ever read. Today, Dex thinks this problem wouldn’t even take four months to develop, now that newer models produce code a lot faster than a year ago. 3. Today’s coding models are most likely trained in a way that makes codebases worse over time. Dex believes that the reason we see LLMs “degrade” existing codebases is because they are optimized to do well on SWE-bench-style benchmarks. These benchmarks reward reproducing a known fix in codebases like Django, but cannot measure poor architecture decisions. This is because the cost function of bad architecture and bad program design cannot be evaluated by running a unit test. It’s a tricky problem to solve: Dex’s best guess is an eval of a model building 20 features in a row in a codebase without knowing what’s coming next. 4. Context engineering 101: figure out where the “dumb zone” begins. As a rule of thumb, the less of the context window that is used, the better the outcomes are. This is because the attention mechanism is quadratic https://newsletter.pragmaticengineer.com/i/141865286/scalability-challenge-from-self-attention : the more that goes into the context window, the more compute is required to process it all. We cover more about self-attention scalability challenges in our ChatGPT deepdive. For a model with a 1M context window, Dex pushes it to around 300-400K when it feels right. For smaller models, he stops at around 100K. You hit the “dumb zone” when its performance starts to degrade because the context window fills up beyond this heuristic limit, and the model begins doing increasingly stupid things like deleting your .env file, for example. 5. A larger context window does not mean a smarter model. Models’ intelligence is behind the ability to use the tokens in the context window, by deciding which parts of the context are relevant for the next decision. You have to get a feel for how much context usage makes sense, and experiment when you hit the “dumb zone.” 6. Frequent, intentional compaction is a technique Dex uses for more complex projects. He will take a long and noisy context, compress it into a Markdown document, then start a new session fresh, pointing the model to this “compressed context” that is in the Markdown. A workflow he uses: One session reads a ton of code filling up its context window while in the “smart zone” , then emits a research document The next session takes tickets describing the work to be done and turns it into a design document The following session takes both documents to create a plan The human is in the loop where it really matters: in this case, reviewing the design document and architecture because Dex finds models to be weak on this 7. Don’t bother optimizing LLM usage until business is booming, there’s massive scale, or high costs. Dex suggests to always start building software with the smartest available model to solve the problem, since engineering time is almost always the bottleneck. Begin to optimize LLM usage and context usage only when at real scale and costs are high enough. That’s when it can be worth using GPT-OSS-120B 1/1,000th the cost of Opus for the simpler steps in your process. 8. “You’re completely right ” or “you’re right to push back on that” are phrases that mean it’s time to start a new session. These responses mean the LLM session is trajectory-poisoned, and you’re wasting time and tokens to continue. Models are autoregressive https://en.wikipedia.org/wiki/Autoregressive model , so if you get into this loop of: Model makes a mistake → user “yells” → model keeps making mistakes → user “yells” … the model calculates that the next most probable message is to make another mistake 9. Only four things matter in the context window: Size : the bigger it is, the more space you should have before hitting the “dumb zone” Information quality . Once something is in the context window, every subsequent turn treats it as fact. This is why errors can compound. Missing information : if there’s information missing that the agent would need, the outcome will be worse, as the agent fills in the gap with guesses. Trajectory. Models are autoregressive, so they predict the next message in the conversation based on previous ones in a kind of thread of reasoning. “Trajectory poisoning” is when the agent gets into a pattern of doing things you don’t want. In this case, it’s time to start over. 10. Slow loops are Dex’s favorite way to do ‘loop engineering.’ The HumanLayer team started with a nightly automation setup that kicks off an agent to fix one thing in the codebase, and open a pull request. In the mornings, they woke up to a PR waiting to be merged. They tweaked it over time and now have four agents open a total of four PRs by the morning, with the focus on code quality improvements. A person still reads all of them before merging. 11. “Token harder” vs. “token smarter”: Dex is in a group chat named ‘Hyper Engineering’, where members share advice on how to max out their Claude subscriptions. This approach, he calls “token harder”. On the other side is “token smarter”: aiming to get maximum value from AI while keeping control. Smarter is harder to pull off. 12. Three ways to run a “software factory.” Here’s options Dex sees as viable: “Turn the lights off: ” go all-in on agentic coding, do not review the code, and pray that AI doesn’t create too much slop. Dex tried this and failed. Read and review all AI-generated code. This slows things down to human speed. Dex says that this way, you should expect a 30-50% lift in productivity from AI, compared to pre-AI engineering. Find leverage, but keep people in the loop. Find out where an hour spent in planning could save four hours’ worth of implementation, in terms of fewer bugs. Invest more time in areas with leverage: design, architecture, and key decisions. Then, let the agent generate code and don’t insist on reviewing all of it. In this way, Dex believes you can move 2-3x faster than when devs wrote all code by hand. The Pragmatic Engineer deepdives relevant for this episode • How Uber uses AI for development: inside look https://newsletter.pragmaticengineer.com/p/how-uber-uses-ai-for-development • Are AI agents actually slowing us down? https://newsletter.pragmaticengineer.com/p/are-ai-agents-actually-slowing-us • AI Tooling for Software Engineers in 2026 https://newsletter.pragmaticengineer.com/p/ai-tooling-2026 • Vibe Coding as a software engineer https://newsletter.pragmaticengineer.com/p/vibe-coding-as-a-software-engineer • AI Engineering in the real world https://newsletter.pragmaticengineer.com/p/ai-engineering-in-the-real-world • How AI-assisted coding will change software engineering: hard truths https://newsletter.pragmaticengineer.com/p/how-ai-will-change-software-engineering • The creator of OpenClaw: “I ship code I don’t read” https://newsletter.pragmaticengineer.com/p/the-creator-of-clawd-i-ship-code Timestamps 00:00 https://www.youtube.com/watch?v=Usufn8IQJgw Intro 01:33 https://www.youtube.com/watch?v=Usufn8IQJgw&t=93s Dex’s path into tech 03:34 https://www.youtube.com/watch?v=Usufn8IQJgw&t=214s Early work in platform engineering 05:28 https://www.youtube.com/watch?v=Usufn8IQJgw&t=328s Replicated 11:24 https://www.youtube.com/watch?v=Usufn8IQJgw&t=684s Metalytics 12:36 https://www.youtube.com/watch?v=Usufn8IQJgw&t=756s 12-factor agents 18:27 https://www.youtube.com/watch?v=Usufn8IQJgw&t=1107s Context engineering 23:38 https://www.youtube.com/watch?v=Usufn8IQJgw&t=1418s Harness engineering 26:11 https://www.youtube.com/watch?v=Usufn8IQJgw&t=1571s Context overload 30:45 https://www.youtube.com/watch?v=Usufn8IQJgw&t=1845s Loop engineering 44:34 https://www.youtube.com/watch?v=Usufn8IQJgw&t=2674s Software factories before and after AI 50:33 https://www.youtube.com/watch?v=Usufn8IQJgw&t=3033s Automation limits 55:18 https://www.youtube.com/watch?v=Usufn8IQJgw&t=3318s Three options for automating 59:00 https://www.youtube.com/watch?v=Usufn8IQJgw&t=3540s RPI framework 1:04:16 https://www.youtube.com/watch?v=Usufn8IQJgw&t=3856s Intentional compaction 1:11:48 https://www.youtube.com/watch?v=Usufn8IQJgw&t=4308s Token harder vs. token smarter 1:16:44 https://www.youtube.com/watch?v=Usufn8IQJgw&t=4604s AI slop 1:19:15 https://www.youtube.com/watch?v=Usufn8IQJgw&t=4755s HumanLayer 1:29:09 https://www.youtube.com/watch?v=Usufn8IQJgw&t=5349s Book recommendation References Where to find Dex Horthy: • LinkedIn: linkedin.com/in/dexterihorthy http://linkedin.com/in/dexterihorthy • Website: https://www.humanlayer.dev https://www.humanlayer.dev Mentions during the episode: • HumanLayer: https://www.humanlayer.dev https://www.humanlayer.dev • Jet Propulsion Laboratory JPL : https://www.jpl.nasa.gov https://www.jpl.nasa.gov • Dykstra’s projection algorithm: https://en.wikipedia.org/wiki/Dykstra’s projection algorithm https://en.wikipedia.org/wiki/Dykstra's projection algorithm • Replicated: https://www.replicated.com https://www.replicated.com • Docker: https://www.docker.com https://www.docker.com • HashiCorp: https://www.hashicorp.com https://www.hashicorp.com • DataStax: https://www.ibm.com/products/datastax https://www.ibm.com/products/datastax • Puppet: https://www.puppet.com https://www.puppet.com • Travis CI: https://www.travis-ci.com https://www.travis-ci.com • Circle CI: https://circleci.com https://circleci.com • Randy Newman’s website: https://www.randynewman.com https://www.randynewman.com • 12-Factor Agents - Principles for building reliable LLM applications: https://github.com/humanlayer/12-factor-agents https://github.com/humanlayer/12-factor-agents • The creator of Clawd: “I ship code I don’t read”: https://newsletter.pragmaticengineer.com/p/the-creator-of-clawd-i-ship-code https://newsletter.pragmaticengineer.com/p/the-creator-of-clawd-i-ship-code • Vaibhav Gupta on LinkedIn: https://www.linkedin.com/in/vaigup https://www.linkedin.com/in/vaigup • Tobi Lutke’s post on X about context engineering: • Andrej Karpathy’s post on X about context engineering: • Improving Deep Agents with harness engineering: https://www.langchain.com/blog/improving-deep-agents-with-harness-engineering https://www.langchain.com/blog/improving-deep-agents-with-harness-engineering • Skill Issue: Harness Engineering for Coding Agents: https://www.humanlayer.dev/blog/skill-issue-harness-engineering-for-coding-agents https://www.humanlayer.dev/blog/skill-issue-harness-engineering-for-coding-agents • Harness engineering for coding agent users: https://martinfowler.com/articles/harness-engineering.html https://martinfowler.com/articles/harness-engineering.html • How AI will change software engineering – with Martin Fowler: https://newsletter.pragmaticengineer.com/p/martin-fowler https://newsletter.pragmaticengineer.com/p/martin-fowler • Dex’s post on X about context reality check: • Laurie Voss on LinkedIn: https://www.linkedin.com/in/seldo https://www.linkedin.com/in/seldo • Everything is a ralph loop: https://ghuntley.com/loop https://ghuntley.com/loop • Dex’s post on X about feedback loops: • Dex’s post on X about token harder vs. token smarter: • Dex’s post on X about AI slop: • GitHub: https://github.com https://github.com • Paul Graham, Live from Stockholm: https://www.ycombinator.com/library/Q7-paul-graham-live-from-stockholm https://www.ycombinator.com/library/Q7-paul-graham-live-from-stockholm • Refactoring: Improving the Design of Existing Code : https://www.amazon.com/dp/0134757599 https://www.amazon.com/dp/0134757599 • Clean Code: A Handbook of Agile Software Craftsmanship : https://www.amazon.com/dp/0132350882 https://www.amazon.com/dp/0132350882 • The Pragmatic Programmer: From Journeyman to Master : https://www.amazon.com/dp/020161622X https://www.amazon.com/dp/020161622X — Production and marketing by Pen Name https://penname.co/ .