Deep Dive: Anthropic's Performance Take-Home (The One Claude Beat Humans At)

Anthropic open-sourced its original performance engineering take-home, which asks candidates to optimize a kernel on a custom VLIW SIMD processor simulator. Claude Opus 4.5 achieved a 99x speedup, reducing the baseline from 147,734 cycles to 1,487 cycles, outperforming most human candidates. The task involves packing operations into instruction bundles to utilize the processor's 12 ALUs, 6 VALUs, and limited memory bandwidth.

Today, Anthropic open-sourced their original performance engineering take-home https://github.com/anthropics/original performance takehome . The task: optimize a kernel running on a custom VLIW https://en.wikipedia.org/wiki/Very long instruction word SIMD https://en.wikipedia.org/wiki/Single instruction, multiple data processor simulator. The baseline takes 147,734 cycles . Claude Opus 4.5 got it down to 1,487 cycles - a 99x speedup that beat most humans. I’m Tristan https://github.com/trirpi @trirpi https://twitter.com/trirpi , and I work on AI kernels. Let’s break down how this whole system works. The Architecture at a Glance the-architecture-at-a-glance This is a VLIW Very Long Instruction Word Single Instruction Multiple Data processor with a SIMD https://en.wikipedia.org/wiki/Single instruction, multiple data single core older versions of the take-home had multiple cores . Let me break down what that means. VLIW: Compiler-Scheduled Parallelism vliw-compiler-scheduled-parallelism In a traditional processor, hardware figures out at runtime which instructions can run in parallel. In a VLIW processor, that job shifts to the compiler or in this case, you . The single core has multiple functional units that can all execute simultaneously: | Unit | Count | Operations | |---|---|---| | ALU | 12 | Scalar: + , - , , / , ^ , & , | , << , , % , < , == | | VALU | 6 | Vector 8 elements : same ops as ALU | | LOAD | 2 | load , vload 8 words , const | | STORE | 2 | store , vstore 8 words | | FLOW | 1 | select , jump , cond jump , halt | You pack operations into instruction bundles . Each cycle, the processor executes one bundle, dispatching operations to all the units in parallel. If you only put one operation in a bundle, the other units sit idle. That’s why the baseline is so slow. Example bundle executes in 1 cycle : {"alu": op1, op2, op3 , "valu": vop1, vop2 , "load": ld1, ld2 } With 12 ALUs and 6 VALUs each processing 8 elements , this single core can theoretically do 12 + 6×8 = 60 arithmetic operations per cycle. Memory Hierarchy memory-hierarchy flowchart LR subgraph mem "💾 MAIN MEMORY" DATA "Problem Data tree, indices, values " end subgraph scratch "📦 SCRATCH SPACE 1536 words " REG "Works like registersAll ALU ops read/write here" end mem <-- |"LOAD/STORE⚠️ 2 each per cycle"| scratch Main Memory : Where the problem data lives. ALU/VALU can’t access it directly. Scratch Space : 1536 words of fast storage. All compute operations read/write scratch addresses. Bottleneck : Only 2 loads and 2 stores per cycle. This is often the limiting factor, not compute. The Execution Engines the-execution-engines The processor has multiple engines , each capable of executing multiple slots per cycle. From problem.py https://github.com/anthropics/original performance takehome/blob/main/problem.py L48-L55 : SLOT LIMITS = { "alu": 12, 12 scalar ALU operations per cycle "valu": 6, 6 vector ALU operations per cycle "load": 2, 2 load operations per cycle "store": 2, 2 store operations per cycle "flow": 1, 1 flow control operation per cycle "debug": 64, Debug operations not counted } What an Instruction Bundle Looks Like what-an-instruction-bundle-looks-like flowchart LR subgraph bundle "📦 Instruction Bundle 1 clock cycle " subgraph compute "Compute" ALU "alu: '+', dest, a, b '-', dest, a, b ' ', dest, a, b ...up to 12" VALU "valu: ' ', vdest, va, vb '+', vdest, va, vb ...up to 6" end subgraph memory "Memory" LOAD "load: 'load', dest, addr 'vload', vdest, addr " STORE "store: 'store', addr, src 'vstore', addr, vsrc " end subgraph control "Control" FLOW "flow: 'select', d, c, a, b " DEBUG "debug: 'compare', loc, key not counted " end end An instruction is a Python dict mapping engine names to lists of operations. Here’s a real example https://github.com/anthropics/original performance takehome/blob/main/problem.py L86-L87 : {"valu": " ", 4, 0, 0 , "+", 8, 4, 0 , "load": "load", 16, 17 } This executes three operations in one cycle : - Vector multiply: scratch 4:12 = scratch 0:8 scratch 0:8 - Vector add: scratch 8:16 = scratch 4:12 + scratch 0:8 - Scalar load: scratch 16 = memory scratch 17 The Problem: Batched Tree Traversal the-problem-batched-tree-traversal The kernel implements a batched tree traversal with hashing. Here’s the flow: php flowchart LR subgraph rounds "🔄 16 Rounds" R0 "Round 0" -- R1 "Round 1" -- R2 "Round 2" -- RN "..." end subgraph batch "📊 Batch of 256 items" B0 "Item 0" B1 "Item 1" B2 "Item 2" BN "..." end subgraph ALGO "⚙️ Per-item computation" A1 "idx = indices i " -- A2 "val = values i " A2 -- A3 "node val = tree idx " A3 -- A4 "val = hash val ^ node val " A4 -- A5 "idx = 2 idx + 1 if even else 2 " A5 -- A6 "if idx = n nodes: idx = 0" end rounds -- batch batch -- ALGO From the reference kernel https://github.com/anthropics/original performance takehome/blob/main/problem.py L467-L484 : python def reference kernel t: Tree, inp: Input : """ A parallel tree traversal where at each node we set cur inp val = myhash cur inp val ^ node val and then choose the left branch if cur inp val is even. If we reach the bottom of the tree we wrap around to the top. """ for h in range inp.rounds : for i in range len inp.indices : idx = inp.indices i val = inp.values i val = myhash val ^ t.values idx idx = 2 idx + 1 if val % 2 == 0 else 2 idx = 0 if idx = len t.values else idx inp.values i = val inp.indices i = idx Test configuration: Tree height : 10 2047 nodes in a perfect binary tree https://www.programiz.com/dsa/perfect-binary-tree Batch size : 256 items processed Rounds : 16 iterations That’s 256 × 16 = 4096 traversal steps, each involving a hash computation. The Hash Function the-hash-function The hash runs 6 stages, each doing a = a op1 const op2 a op3 shift : | Stage | Formula | |---|---| | 0 | a = a + 0x7ED55D16 + a << 12 | | 1 | a = a ^ 0xC761C23C ^ a 19 | | 2 | a = a + 0x165667B1 + a << 5 | | 3 | a = a + 0xD3A2646C ^ a << 9 | | 4 | a = a + 0xFD7046C5 + a << 3 | | 5 | a = a ^ 0xB55A4F09 ^ a 16 | Each stage = 3 ALU ops. Total: 6 × 3 = 18 ALU operations per hash. The hash is defined data-driven https://github.com/anthropics/original performance takehome/blob/main/problem.py L439-L464 for easy kernel implementation: HASH STAGES = "+", 0x7ED55D16, "+", "<<", 12 , "^", 0xC761C23C, "^", " ", 19 , "+", 0x165667B1, "+", "<<", 5 , "+", 0xD3A2646C, "^", "<<", 9 , "+", 0xFD7046C5, "+", "<<", 3 , "^", 0xB55A4F09, "^", " ", 16 , Similar to Bob Jenkins’ hash https://en.wikipedia.org/wiki/Jenkins hash function . Each stage: a = a op1 val1 op2 a op3 val3 The Memory Model the-memory-model Two memory spaces: Main Memory self.mem : Problem input/output Scratch Space core.scratch : 1536 words - think of it as registers + constant memory + manually managed cache From the constants https://github.com/anthropics/original performance takehome/blob/main/problem.py L57-L60 : VLEN = 8 Vector length: 8 elements N CORES = 1 Single core older versions had multiple SCRATCH SIZE = 1536 1536 words of scratch space Every ALU operation reads and writes scratch addresses. It’s like programming a GPU with shared memory https://developer.nvidia.com/blog/using-shared-memory-cuda-cc/ , but more explicit. The ALU Operations the-alu-operations python def alu self, core, op, dest, a1, a2 : a1 = core.scratch a1 a2 = core.scratch a2 match op: case "+": res = a1 + a2 case "-": res = a1 - a2 case " ": res = a1 a2 case "//": res = a1 // a2 case "^": res = a1 ^ a2 XOR case "&": res = a1 & a2 AND case "|": res = a1 | a2 OR case "<<": res = a1 << a2 Left shift case " ": res = a1 a2 Right shift case "%": res = a1 % a2 Modulo case "<": res = int a1 < a2 Comparison case "==": res = int a1 == a2 res = res % 2 32 32-bit unsigned wrap self.scratch write dest = res Vector ALU ops https://github.com/anthropics/original performance takehome/blob/main/problem.py L254-L267 - the SIMD part: python def valu self, core, slot : match slot: case "vbroadcast", dest, src : Broadcast scalar to all 8 vector lanes for i in range VLEN : self.scratch write dest + i = core.scratch src case "multiply add", dest, a, b, c : Fused multiply-add: dest = a b + c for i in range VLEN : mul = core.scratch a + i core.scratch b + i % 2 32 self.scratch write dest + i = mul + core.scratch c + i % 2 32 case op, dest, a1, a2 : Any scalar op applied element-wise for i in range VLEN : self.alu core, op, dest + i, a1 + i, a2 + i Memory Operations memory-operations Load/store https://github.com/anthropics/original performance takehome/blob/main/problem.py L269-L298 - only 2 of each per cycle: python def load self, core, slot : match slot: case "load", dest, addr : self.scratch write dest = self.mem core.scratch addr case "vload", dest, addr : 8 consecutive elements addr = core.scratch addr for vi in range VLEN : self.scratch write dest + vi = self.mem addr + vi case "const", dest, val : self.scratch write dest = val % 2 32 Key bottleneck: only 2 loads per cycle . Vector loads vload help - 8 elements in one slot Flow Control flow-control Flow ops https://github.com/anthropics/original performance takehome/blob/main/problem.py L300-L335 - crucial for branchless programming https://en.algorithmica.org/hpc/pipelining/branchless/ : python def flow self, core, slot : match slot: case "select", dest, cond, a, b : Branchless: dest = cond ? a : b self.scratch write dest = core.scratch a if core.scratch cond = 0 else core.scratch b case "vselect", dest, cond, a, b : Vector version for vi in range VLEN : self.scratch write dest + vi = core.scratch a + vi if core.scratch cond + vi = 0 else core.scratch b + vi case "cond jump", cond, addr : if core.scratch cond = 0: core.pc = addr case "jump", addr : core.pc = addr Why the Baseline Is So Slow why-the-baseline-is-so-slow The baseline kernel https://github.com/anthropics/original performance takehome/blob/main/perf takehome.py L88-L175 deliberately uses one operation per cycle : python def build self, slots: list tuple Engine, tuple , vliw: bool = False : Simple slot packing that just uses one slot per instruction bundle instrs = for engine, slot in slots: instrs.append {engine: slot } One op per bundle return instrs So instead of: {"alu": op1, op2, op3 , "load": load1 } 1 cycle You get: {"alu": op1 } Cycle 1 {"alu": op2 } Cycle 2 {"alu": op3 } Cycle 3 {"load": load1 } Cycle 4 4 cycles instead of 1. The 12 ALU slots sit empty. Debugging Tools debugging-tools Perfetto Trace Viewer perfetto-trace-viewer The simulator outputs Chrome Trace Event Format https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview traces viewable in Perfetto https://ui.perfetto.dev/ : python perf takehome.py Tests.test kernel trace python watch trace.py Opens browser with live-reloading trace The watch trace.py https://github.com/anthropics/original performance takehome/blob/main/watch trace.py server auto-reloads traces when they change - great for iterating. Debug Instructions debug-instructions Debug ops https://github.com/anthropics/original performance takehome/blob/main/problem.py L365-L382 verify intermediate values without counting cycles: body.append "debug", "compare", tmp val, round, i, "hashed val" Optimization Strategies optimization-strategies The key techniques: 1. VLIW Packing 1-vliw-packing Pack independent operations into the same cycle: {"alu": op1, op2, op3 , "load": load1, load2 } 5 ops, 1 cycle 2. SIMD Vectorization 2-simd-vectorization Process 8 batch items at once with valu and vload / vstore . 3. Software Pipelining https://en.wikipedia.org/wiki/Software pipelining 3-software-pipelining Overlap computation of different iterations to keep all engines busy. 4. Branchless with select 4-branchless-with-select Instead of conditional jumps: offset = select val % 2 == 0, 1, 2 idx = 2 idx + offset The Hard Part: Data Dependencies the-hard-part-data-dependencies You can’t compute the next tree index until you’ve hashed the current value. That’s a serial dependency chain https://en.wikipedia.org/wiki/Instruction-level parallelism Data dependency . The trick is finding parallelism across different batch items. Further Reading further-reading VLIW Architecture - Wikipedia https://en.wikipedia.org/wiki/Very long instruction word SIMD Programming - Intel Intrinsics Guide https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html Software Pipelining - Wikipedia https://en.wikipedia.org/wiki/Software pipelining Perfetto UI https://ui.perfetto.dev/ Chrome Trace Event Format https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview Computer Architecture: A Quantitative Approach https://www.amazon.com/Computer-Architecture-Quantitative-Approach-Kaufmann/dp/0128119055 Branchless Programming https://en.algorithmica.org/hpc/pipelining/branchless/