A C File Runs a 744B Model, Cloudflare Maps Agentic Traffic, and Karpathy's $48 GPT-2 - The Tokenizer Edition #36

The Tokenizer newsletter #36 reports that a single C file can stream a 744-billion-parameter model from disk, Cloudflare has mapped agentic traffic patterns, and Andrej Karpathy's nanochat trains a GPT-2-class model for about $48 on eight H100s. The issue also highlights five papers, including RST, which generated over 37,000 tasks and improved Terminal-Bench Hard performance by 41.16% over baseline.

Hey there This week, one C file streams a 744-billion-parameter model from disk, Cloudflare maps how humans and agents share web sessions, and Karpathy’s nanochat trains a GPT-2-class model for about $48 on eight H100s. New here? The Tokenizer is where I collect the best AI/ML papers, videos, articles, tools, and learning resources, so you can skip the noise. Subscribe to Gradient Ascent https://newsletter.artofsaience.com . TL;DR What caught my attention this week: - 📄 Papers: Harder terminal tasks, replayable screen memory, editable 3D worlds, training without an external simulator, and a map of agentic economies. - 🎥 Videos: Serving open models, Linear’s Slack-to-PR agent, a 3D attention demo, and why voice models stumble on timers and spelling. - 📰 Reads: Cloudflare’s map of agentic traffic, an early Apple SpeechAnalyzer benchmark, and a Google DeepMind researcher on delegation and agency. - 🛠 Tools: One C file that runs a 744B model from disk, plus a persistent code graph that supports more than 150 languages. - 🎓 Learning: Train a GPT-2-class model for about $48 and trace the whole pipeline from tokenizer to chat. 📄 5 Papers 1. Recursive Synthesis for Long-Horizon Terminal Tasks https://arxiv.org/abs/2608.05466 https://arxiv.org/abs/2608.05466 Terminal agents need training tasks that keep getting harder as the model improves. RST starts with verified seed tasks, generates tougher variants, checks them in a sandbox, and uses the successful ones as new starting points. That loop produced more than 37,000 tasks and lifted Terminal-Bench Hard performance by 41.16% over the baseline, without researchers having to write every exercise by hand. 2. Activity Frames: Deterministic Screen-Activity Compilation for Agent Memory and Replay https://arxiv.org/abs/2608.05784 https://arxiv.org/abs/2608.05784 | GitHub https://github.com/nossa-y/activity-frames Agents that watch your screen can burn tokens rediscovering routines they have already seen. Activity Frames records events, session boundaries, and accessibility-tree context, then compiles them into memory the agent can replay. On one representative day, it cut the record from 126,812 tokens of serialized JSON to 1,469 tokens. The much larger headline ratio is modeled from one user, but the underlying advantage is clear: structured memory is cheaper to revisit than a full serialized log. 3. WorldClaw: Agentic 3D Open-World Generation at Scale https://arxiv.org/abs/2608.05248 https://arxiv.org/abs/2608.05248 Give WorldClaw a prompt and it builds an explorable 3D scene that remains editable object by object. A planning agent breaks the request into assets, terrain, and layout; Blender assembles the scene; rendered views guide another round of refinement. The examples look promising, but the paper evaluates them qualitatively rather than against a benchmark. 4. EnvACE: Internalizing Environment Dynamics via World Rehearsal https://arxiv.org/abs/2608.06197 https://arxiv.org/abs/2608.06197 | GitHub https://github.com/Within-yao/EnvACE EnvACE trains one model to both make a tool call and predict what the environment will return. Because the same reward signal improves both jobs, the model can rehearse interactions without calling an external simulator every time. The overall gains are modest, at 0.99 and 0.37 points, and final testing still uses the real environment. The potential saving is in training, not deployment. 5. From Economic Agents to Agentic Economies: A Systems Blueprint https://arxiv.org/abs/2608.06020 https://arxiv.org/abs/2608.06020 | GitHub https://github.com/FreedomIntelligence/Awesome-Economic-World-Models Research on agentic economies still mostly studies simple worlds with fixed rules. After reviewing 881 papers and validating 737, the authors find that ambitious ideas such as self-modifying agents and evolving institutions remain thinly studied. That makes the paper a good map of the field, though it cannot tell us whether those missing pieces would make agent systems safer. 🎥 4 Videos 1. What Happens Between an Open Model Release and Production Speed Open weights are only the start of a fast production API. Philip Kiely and Ali Taha show the work that follows: cache-aware routing, separating prefill from decode, speculative decoding, and kernel tuning. Some changes doubled or tripled throughput, though the gains are from separate tests rather than one cumulative result. Day-one model support still depends on engineers profiling the stack and replacing slow layers. 2. How Linear Turns a Slack Message Into a PR in Six Minutes Linear’s internal agent can turn a Slack request into a pull request in about six minutes. Nan Yu and Jacob Shumway first narrowed the job in a memo, then gave the agent tools to fetch context instead of cramming it into the prompt. They added evals as adoption grew. The demo stops at the PR, but the build sequence is solid: define a narrow task, start with the strongest model you can use, and add evals before a broad rollout. 3. Building a Tiny Attention Head in Three Dimensions Luis Serrano starts with a two-dimensional attention setup that cannot produce the relationships he wants, then moves to three dimensions and gets a working construction. From there, he uses that construction to explain the geometry and shares runnable code you can test before tackling the algebra. It shows one path that works without claiming every 2D route must fail. 4. Why Voice AI Struggles With Timers In these tests, ChatGPT Voice struggles with timers, letter counting, and deliberate silence. Julia Turc and router.audio founder Jiameng Gao work through several possible causes, including audio tokenization, model routing, tool access, and how the voice stack tracks time. None is established as the single explanation, but the recurring failures make the system’s weak spots easy to see. 📰 3 Curated Reads 1. Unveiling Good and Bad Behaviors on the Agentic Internet https://blog.cloudflare.com/good-and-bad-agentic-behaviors/ https://blog.cloudflare.com/good-and-bad-agentic-behaviors/ Cloudflare recorded 206 million Precursor evaluations in a day and found that a web session can shift between human and automated control. That makes one-time bot checks too blunt: the risk can change after a session begins, and trust has to be reassessed over time. If you run a web product, assume some sessions already mix human and agentic actions. 2. Apple’s SpeechAnalyzer vs Whisper: An Early Benchmark https://get-inscribe.com/blog/apple-speech-api-benchmark.html https://get-inscribe.com/blog/apple-speech-api-benchmark.html Lyonesse tested Apple’s SpeechAnalyzer against Whisper Small on LibriSpeech test-clean. SpeechAnalyzer posted a 2.12% word error rate versus 3.74%, with preliminary timing around three times faster. It is one vendor-run benchmark, so treat it as a reason to test SpeechAnalyzer on your own audio rather than a final verdict. 3. Are We Offloading Too Much of Our Thinking to AI? Before you prompt, ask whether you need help carrying out a choice or making it. Yennie Jun, a Google DeepMind researcher working on Gemini, separates handing off the work from handing off the decision. AI can be useful once you know what you want; the risk begins when you hand over that judgment too. 🛠 2 Tools & Repos 1. JustVugg/colibri https://github.com/JustVugg/colibri https://github.com/JustVugg/colibri Colibri runs a 744-billion-parameter mixture-of-experts model from a single dependency-free C file. It treats VRAM, RAM, and NVMe as tiers of one memory system, keeping dense layers in RAM and pulling selected experts from disk as needed. The tradeoff is speed, but the result is enough capacity to run a huge model on a desktop. 2. DeusData/codebase-memory-mcp https://github.com/DeusData/codebase-memory-mcp https://github.com/DeusData/codebase-memory-mcp Point codebase-memory-mcp at a repository and it builds a persistent knowledge graph with tree-sitter. It covers more than 150 languages and answers questions about symbols, call paths, and dead code without rescanning every file. Because it ships as a single static binary, there is no Docker setup, language server, or API key to manage. 🎓 1 Pick of the Week karpathy/nanochat https://github.com/karpathy/nanochat https://github.com/karpathy/nanochat For about $48 on eight H100s, nanochat trains a GPT-2-class model you can actually chat with. Set --depth and the code derives the width, learning rate, and schedule for your compute budget. The repository is compact enough to follow end to end, from tokenizer training to the final conversation, so you can see how the whole stack fits together. Thanks for reading. If this issue helped, send it to a colleague who’d enjoy it, or subscribe to Gradient Ascent https://newsletter.artofsaience.com .