An Illustrated Guide to AI Agents

Authors Maarten Grootendorst and Jay Alammar released the companion code repository for their book "An Illustrated Guide to AI Agents," which walks readers through building a "TinyAgent" from scratch using only LLM calls. The book spans 10 chapters—from an introduction to AI agents and large language models through memory, tools, planning, evaluation, multi-agent collaboration, multimodal understanding and coding agents—and contains more than 300 custom figures, with all examples runnable for free on Google Colab's T4 GPU. The repository adds free visual guides on topics including Gemma 4 and multimodal generation, and the book drew endorsements from Chip Huyen, MIT EECS assistant professor Omar Khattab and Meta FAIR research scientist Ofir Press.

Welcome Here, you will find the code for the book An Illustrated Guide to AI Agents https://www.amazon.com/Illustrated-Guide-AI-Agents-Concepts/dp/B0GTYL2QSJ written by Maarten Grootendorst https://www.linkedin.com/in/mgrootendorst/ and Jay Alammar https://www.linkedin.com/in/jalammar/ . In this repository, you will: "Build an Agent From Scratch " Through the visually educational nature of this book and with more than 300 custom made figures , learn the practical tools and concepts you need to use AI Agents today The code examples in this repo are used for you to start building a TinyAgent entirely from scratch, using nothing more than LLM calls. You will learn how to create your own Agent that has memory, tools, and autonomy All examples can be run in Google Colab for free using their T4 GPU. You have options for running everything either locally or on the cloud without any additional costs involved. | Chapter | Notebook | |---|---| | Chapter 1: Introduction to AI Agents | | | Chapter 2: Large Language Models | | | Chapter 3: Reasoning Large Language Models | | | Chapter 4: Memory | | | Chapter 5: Tools | | | Chapter 6: Planning | | | Chapter 7: Evaluation | | | Chapter 8: Multi-Agent Collaboration | | | Chapter 9: Multimodal Understanding | | | Chapter 10: Coding Agents | | Tip You can check the setup https://github.com/HandsOnLLM/An-Illustrated-Guide-To-AI-Agents/blob/main/.setup folder for a quick-start guide to install all packages locally. " The authors are exceptional builders, and this book is a testament to the depth of their knowledge. The generous illustrations and hands-on TinyAgent exercise make the concepts approachable, intuitive, and fun to learn. " Chip Huyen - author of AI Engineering https://www.oreilly.com/library/view/ai-engineering/9781098166298/ and Designing Machine Learning Systems https://www.oreilly.com/library/view/designing-machine-learning/9781098107956/ " I have thought a lot about how to teach AI systems, and this book still surprised me with how much of the domain of modern AI agents it could make clear through illustrations. Maarten and Jay do an impressive job identifying and explaining the timeless ideas in this fast-moving field, and I expect this to become the first book I recommend to anyone who wants to understand how modern agents work and how to build them. " Omar Khattab - assistant professor, MIT EECS. " Grootendorst and Alammar explain the entire AI agent stack, from tokenization to multiagent systems, in an engaging visual style. As the field evolves, this book will continue to serve as a useful reference because it covers the subject matter so thoroughly. " Ofir Press - research scientist at Meta FAIR, coauthor of SWE-bench and SWE-agent. We attempted to put as much information into the book without it being overwhelming. However, even with more than 300 visuals there is still much to discover We continue to create free visual guides that complement the book and go more in-depth into new and exciting topics https://github.com/HandsOnLLM/An-Illustrated-Guide-To-AI-Agents/blob/main/bonus : | A Visual Guide to Gemma 4 https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-gemma-4 | A Visual Guide to Multimodal Generation https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-multimodal-generation | |---|---| We wanted to do something special this time around and allow readers to Build an Agent From Scratch However, we did not stop there and wanted the act of building the Agent to be a modular experience that enhances the learning experience. By iteratively adding components, one at a time, it becomes much more intuitive how an Agent actually works. We decided to combine this with creative if we say so ourselves ways to explain what is happening and how each added component affects your TinyAgent Through a philosophy centered around modularity, each chapter can neatly cover a single topic. As such, that allows us to do things like annotate code to help you understand what reason is for specific lines of code. As we are building up your TinyAgent there might be many code changes happening in certain chapters. To illustrate the effect of these changes on your TinyAgent , we made use of diffs as a way to ease the learning curve. Please consider citing the book if you consider it useful for your research: @book{illustrated-agents-book, author = {Maarten Grootendorst and Jay Alammar}, title = {An Illustrated Guide to AI Agents}, publisher = {O'Reilly}, year = {2026}, isbn = {979-8341662698}, url = {https://www.oreilly.com/library/view/an-illustrated-guide/9798341662681/}, github = {https://github.com/HandsOnLLM/An-Illustrated-Guide-To-AI-Agents} }