Agentic Kernel Generation

A body of research published between July 2025 and December 2025 describes agentic kernel generation, in which autonomous LLM agents run iterative, feedback-driven loops to synthesize, validate and optimize computational kernels across heterogeneous hardware and DSLs. Systems such as TritorX (Hammond et al., 3 Dec 2025), which targets Meta MTIA with Triton, and GEAK (Wang et al., 31 Jul 2025), which targets AMD MI300X with Triton, pair LLM generators with linters, compilers, test harnesses, evaluators and optimizers to check functional correctness and improve performance. The approach shifts kernel enablement from static human engineering to adaptive closed-loop workflows spanning NVIDIA, AMD, Meta MTIA, NPUs and CPUs, and DSLs including Triton, CUDA, CuTe and TileLang.

Agentic Kernel Generation - Agentic kernel generation is an automated approach that employs autonomous LLM agents in iterative loops to generate, validate, and optimize computational kernels. - It leverages multi-modal feedback—including static analysis, JIT profiling, and empirical tests—to ensure functional correctness and performance improvement. - These systems support heterogeneous hardware and DSLs, achieving significant speedups and broad operator coverage through adaptive, closed-loop workflows. Agentic kernel generation refers to the automated synthesis, validation, and optimization of computational kernels or system-level primitives using agentic workflows—primarily those driven by LLMs operating in iterative, feedback-driven, and often multi-agent loops. These systems transform kernel enablement from a static human-engineered activity to an adaptive, scalable, and context-sensitive process, targeting diverse hardware and software stacks from AI accelerators to OS subsystems Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 , Wang et al., 31 Jul 2025 https://www.emergentmind.com/papers/2507.23194 , Zhang et al., 23 Oct 2025 https://www.emergentmind.com/papers/2511.01884 , Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 , Du et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23424 , Dong et al., 19 Oct 2025 https://www.emergentmind.com/papers/2510.16996 , Zhang et al., 19 Nov 2025 https://www.emergentmind.com/papers/2511.15915 , Zheng et al., 1 Sep 2025 https://www.emergentmind.com/papers/2509.01245 . 1. Core Concepts and Definitions Agentic kernel generation systems are distinguished by their use of autonomous agents—typically implemented via LLMs—which iteratively generate, assess, and refine kernel implementations. Unlike naive prompt-based or one-shot code generation, these systems use closed feedback loops incorporating both programmatic and empirical checks: - Agentic loop : The central workflow, frequently modeled as a finite-state machine https://www.emergentmind.com/topics/finite-state-machine-fsm FSM https://www.emergentmind.com/topics/fast-spatial-memory-fsm or as a search tree or graph traversed by cooperative agents. Stages include code generation, static analysis, compilation, hardware execution, and feedback extraction. - Coverage orientation : Prioritization of correct functional coverage across large kernel/operator sets, supporting all data types, signature patterns, and argument shapes Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 . - Multi-modal feedback : Integration of static program checks linting, AST analysis , dynamic runtime profiling JIT, hardware counters , empirical correctness test harnesses , and knowledge retrieval from documentation or historical experience Zhang et al., 23 Oct 2025 https://www.emergentmind.com/papers/2511.01884 , Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 . - Iterative refinement : Use of LLM-based agents https://www.emergentmind.com/topics/llm-based-agents or subagents specialized for code synthesis, error diagnosis, optimization suggestion, or plan decomposition, operating in feedback loops inspired by human engineering workflows. - Heterogeneous and cross-platform support : Compatibility with multiple hardware backends e.g., NVIDIA, AMD, Meta MTIA, NPUs, CPUs and diverse kernel DSLs https://www.emergentmind.com/topics/domain-specific-languages-dsls Triton, CUDA, CuTe, TileLang Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 , Du et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23424 . 2. Architectures and Methodological Taxonomy Several design archetypes for agentic kernel generation have converged in recent work: | System | Architecture | Agents / Submodules | Hardware / DSLs | |---|---|---|---| | TritorX Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 | FSM per operator | LLM generator, Linter, Compiler, Test harness, Log Summarizer | Meta MTIA, Triton | | GEAK Wang et al., 31 Jul 2025 https://www.emergentmind.com/papers/2507.23194 | Multi-agent pipeline https://www.emergentmind.com/topics/multi-agent-pipeline | Generator, Evaluator, Reflector, Optimizer | AMD MI300X, Triton | | CudaForge https://www.emergentmind.com/topics/cudaforge Zhang et al., 23 Oct 2025 https://www.emergentmind.com/papers/2511.01884 | Two-agent Coder, Judge | Correction, Optimization hardware feedback | CUDA, NVIDIA GPUs https://www.emergentmind.com/topics/graphics-processing-units-gpus | | KernelEvolve https://www.emergentmind.com/topics/kernelevolve Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 | Graph search universal operator | Node selection, Universal operator, Eval, Retriever | NVIDIA, AMD, MTIA. Triton, CuTe, MLIR https://www.emergentmind.com/topics/multilevel-intermediate-representation-mlir | | AKG Du et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23424 | Closed-loop, modular | Designer, Coder, Verifier, Conductor https://www.emergentmind.com/topics/conductor | Triton, CUDA-C, TileLang, CPP | | STARK https://www.emergentmind.com/topics/spatial-transcriptomics-via-adaptive-regularization-and-kernels-stark Dong et al., 19 Oct 2025 https://www.emergentmind.com/papers/2510.16996 | Tree search, multi-agent | Search controller, Plan agent, Code agent, Debug/Profiler | CUDA | | AccelOpt https://www.emergentmind.com/topics/accelopt Zhang et al., 19 Nov 2025 https://www.emergentmind.com/papers/2511.15915 | Beam-search loop | Planner, Executor, Summarizer, Memory | AWS Trainium/NKI | | SchedCP https://www.emergentmind.com/topics/schedcp Zheng et al., 1 Sep 2025 https://www.emergentmind.com/papers/2509.01245 | Multi-agent, decoupled OS | Observation, Planning, Execution, Learning | Linux eBPF, Schedulers | Most implementations structure kernel generation as an iterative process: 1 candidate generation via an LLM often context-conditioned , 2 static or dynamic formal verification, 3 JIT compilation or hardware execution, and 4 response-driven prompt or memory updates. Architectures range from explicit FSMs https://www.emergentmind.com/topics/neural-finite-state-machines-fsms Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 , beam or tree search Zhang et al., 19 Nov 2025 https://www.emergentmind.com/papers/2511.15915 , Dong et al., 19 Oct 2025 https://www.emergentmind.com/papers/2510.16996 , Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 , to multi-agent modular systems Du et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23424 , Wang et al., 31 Jul 2025 https://www.emergentmind.com/papers/2507.23194 , Zhang et al., 23 Oct 2025 https://www.emergentmind.com/papers/2511.01884 . 3. Formal Decision Criteria and Optimization Objectives Agentic kernel generation systems formalize correctness and fitness criteria as binary and continuous objectives grounded in hardware-realized execution: - Lint and static correctness : Candidate passes if all linter rules yield zero violations: - Functional correctness : Operator passes if outputs match a canonical backend within across all relevant test inputs: - Coverage : Fraction of operators or benchmarks with complete pass rates; indicates full correctness. - Performance objectives : Speedup relative to reference implementation, e.g., TritorX’s fitness: - Termination : Completion upon reaching target coverage, improvement stall, or artifact budget exhaustion: Agent selection and expansion often use softmax, -greedy, or Monte Carlo Tree Search https://www.emergentmind.com/topics/monte-carlo-tree-search policies over observed fitness or coverage scores Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 , Dong et al., 19 Oct 2025 https://www.emergentmind.com/papers/2510.16996 . In evaluation, systems report metrics such as median speedup, percent exceeding baseline, pass@K, and per-operator correctness. 4. Feedback Mechanisms and Context Management Effective agentic kernel pipelines depend on multi-level and multi-modal feedback, including: - Static linter/AST analysis blocking unsafe or “cheating” constructs e.g., host fallback, recursive ATen calls Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 . - JIT compile/test failures summarized and filtered for prompt brevity; secondary LLMs often condense error logs Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 , Zhang et al., 23 Oct 2025 https://www.emergentmind.com/papers/2511.01884 . - Empirical profiling : Distributed execution on real hardware FPGA/ASIC/GPU/CPU/NPU , capturing performance counters, occupancy, memory throughput Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 , Wang et al., 31 Jul 2025 https://www.emergentmind.com/papers/2507.23194 , Zhang et al., 23 Oct 2025 https://www.emergentmind.com/papers/2511.01884 . - Retrospective experience/memory : Explicit archives of slow–fast kernel pairs, with summarizing LLMs to extract transferrable transformations Zhang et al., 19 Nov 2025 https://www.emergentmind.com/papers/2511.15915 . - Contextual retrieval : Retrieval-augmented prompts https://www.emergentmind.com/topics/retrieval-augmented-prompts fuse runtime bottlenecks, prior kernel variants, and documentation slices to inform the next iteration Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 , Du et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23424 . Context management strategies https://www.emergentmind.com/topics/context-management-strategies include prompt truncation, focused tokenization e.g., bottleneck-extracted artifacts only , and dynamic context windows specific to each agent’s role planning, coding, debugging Dong et al., 19 Oct 2025 https://www.emergentmind.com/papers/2510.16996 . 5. Empirical Evaluation and Benchmark Results Scalable agentic kernel systems report the following empirical capabilities: - Operator and primitive coverage : TritorX generated correct wrappers for 481/568 PyTorch ATen operators on MTIA 84.7% OpInfo coverage, tests Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 . KernelEvolve achieved correctness on 250 KernelBench https://www.emergentmind.com/topics/kernelbench-cuda problems and 160 ATen operators across three platforms Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 . - Performance : Agentic systems https://www.emergentmind.com/topics/agentic-systems yield consistent speedups. CudaForge attains a median over PyTorch on diverse GPUs Zhang et al., 23 Oct 2025 https://www.emergentmind.com/papers/2511.01884 . KernelEvolve achieves up to 0 on specific tasks, and AKG reports 1 speedup on Triton-CUDA kernels Du et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23424 , Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 . - Efficiency and cost : CudaForge requires only 20.33\sim 26.5$ min wall-clock, substantially below prior agentic baselines Zhang et al., 23 Oct 2025 https://www.emergentmind.com/papers/2511.01884 . - Robustness & generality : Complex operator sets, broad datatypes, model-in-the-loop testing NanoGPT, DLRM, MM1, MM2 , and cross-platform adaptability are directly validated Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 , Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 . - Ablations : Removal of critical agents linter, compilation log summarizer, optimizer degrades coverage and performance significantly Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 , Wang et al., 31 Jul 2025 https://www.emergentmind.com/papers/2507.23194 . 6. Key Design Trade-offs and Future Directions - Coverage-first vs. performance-first : Systems like TritorX and KernelEvolve prioritize broad operator support and functional correctness, often deferring fine-grained autotuning. Others GEAK, AccelOpt explicitly tune for hardware efficiency post-correctness Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 , Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 , Zhang et al., 19 Nov 2025 https://www.emergentmind.com/papers/2511.15915 . - FSM vs. fully agentic execution : FSMs offer stringent control/reproducibility; agentic architectures https://www.emergentmind.com/topics/agentic-architectures with tool-enabling LLMs promise more flexible, adaptive workflows. The field is evolving toward hybrid agentic orchestration https://www.emergentmind.com/topics/agentic-orchestration with tool APIs as first-class interfaces Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 , Du et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23424 . - Context and memory management : Retrieval-augmented prompting https://www.emergentmind.com/topics/retrieval-augmented-prompting and lightweight summarization ensure scalability and cost control; explicit long-term memory https://www.emergentmind.com/topics/long-term-memory-ltm or archive-guided planning accelerates convergence and transfers optimization patterns Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 , Zhang et al., 19 Nov 2025 https://www.emergentmind.com/papers/2511.15915 , Du et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23424 . - Extensibility : Agentic frameworks https://www.emergentmind.com/topics/agentic-frameworks are structured for rapid integration of new hardware backends and DSLs by swapping out DocSpecs, knowledge bases, or hardware constraints in context Du et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23424 , Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 . - Safety and validation : Production agentic deployment https://www.emergentmind.com/topics/agentic-deployment mandates strict anti-cheating policies, comprehensive test harnesses, and staged verification to guarantee correctness under all observed usage Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 , Zhang et al., 23 Oct 2025 https://www.emergentmind.com/papers/2511.01884 . Possible directions include reinforcement or active learning https://www.emergentmind.com/topics/active-learning-al for plan selection, embedding-based retrieval https://www.emergentmind.com/topics/embedding-based-retrieval-ebr for optimization memory, multi-platform joint search, and coordinated agentic optimization https://www.emergentmind.com/topics/agentic-optimization-aaio across kernel, OS, and system stack subsystems Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 , Zheng et al., 1 Sep 2025 https://www.emergentmind.com/papers/2509.01245 , Zhang et al., 19 Nov 2025 https://www.emergentmind.com/papers/2511.15915 . 7. Broader Context and Philosophical Underpinnings The agentic kernel generation paradigm signifies a transition from monolithic, single-shot code generation toward open-ended, self-adaptive, and robust code synthesis systems, moving beyond traditional AI-hardware co-design cycles Hammond et al., 3 Dec 2025 https://www.emergentmind.com/papers/2512.10977 , Liao et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23236 , Du et al., 29 Dec 2025 https://www.emergentmind.com/papers/2512.23424 . The term "agentic kernel" also echoes research in cognitive architectures, where a minimal "functional kernel" enables autonomous emergence of higher-level cognitive functions through reflexive, schema-based self-organization Serov, 2022 https://www.emergentmind.com/papers/2207.00822 . This analogy underscores the trajectory of future agentic kernel platforms: to provide the substrate from which both routine and emergent computation can be self-organized and optimized—potentially closing the last-mile gap in hardware–software co-evolution.