Agent-Assisted SGLang Development: An Initial Exploration SGLang development is being augmented with agent-assisted workflows that encode procedural engineering knowledge into executable skills, covering LLM serving, GPU kernels, diffusion pipelines, and production incident handling. These skills, maintained in repositories like SGLang's .claude/skills and KDA-Pilot, enable reproducible experiments, benchmarking, profiling, and review loops, with agent-derived work already landing in multiple SGLang integration PRs. The approach emphasizes that agents are most valuable when they follow well-defined workflows, and that human review remains critical as agents generate more plausible changes. Agent-Assisted SGLang Development: An Initial Exploration SGLang development increasingly goes beyond isolated code changes. The same repository now spans LLM serving, distributed runtime, GPU kernels, diffusion pipelines, model-specific execution paths, and production incident handling. In the past, many of these workflows depended on individual developer memory: how to launch a certain model, how to read a profile trace, which log to add first when debugging a CUDA crash, or which benchmarks a performance PR should include. As agent tools mature, this experience can be turned into executable SKILL.md files, scripts, benchmark contracts, and review loops. Around SGLang agent development, a set of skills has already emerged for both LLM and diffusion work: SGLang https://github.com/sgl-project/sglang/tree/main/.claude/skills is maintained inside the SGLang repository and captures repo-level development workflows such as CUDA crash debugging, kernel integration, tests, CI, profiling, production triage, and source-tree conventions. .claude/skills SGLang diffusion https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal gen/.claude/skills focuses on diffusion-specific workflows, including adding new diffusion models, benchmarking and profiling denoise paths, tuning performance options, and validating quantized pipelines. .claude/skills BBuf/AI-Infra-Auto-Driven-SKILLS https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS covers cross-framework serving benchmarks, capacity planning, profile and pipeline analysis, model compute simulation, SGLang human-style review, production incident triage, SOTA loops for SGLang and other open-source inference frameworks, and model PR history. kernel-design-agents https://github.com/mit-han-lab/kernel-design-agents is the KDA project and the winning solution for the MLSys 2026 FlashInfer Kernel Contest. BBuf/KDA-Pilot https://github.com/BBuf/KDA-Pilot applies KDA-style agent kernel workflows to SGLang. Its public B200 diffusion summary now tracks 10 SGLang kernel tasks. Most rows come from KDA-Pilot's public benchmark ledger, while residual gate add uses the B200 speedup reported by the merged SGLang integration PR after the original task baseline moved. KDA-Pilot-derived work has now landed in three SGLang integration PRs. Viewed together, these efforts point to the same direction: the value of agents comes from procedural engineering knowledge, including executable steps, reproducible experiments, and reviewable evidence. 1. TL;DR - Agents are most useful in SGLang when they can keep moving along a well-defined workflow. Benchmarking, profiling, kernel API logging, adding diffusion pipelines, production incident replay, and SOTA loops can all be encoded as skills. - An SGLang skill is an executable development procedure. In debug-cuda-crash , sglang-diffusion-benchmark-profile , and llm-torch-profiler-analysis , the important content is preflight checks, hard failure gates, artifact contracts, reproduction commands, and result formats. - Profile evidence is central to performance work. The SGLang profiler skills produce fixed kernel tables, overlap-opportunity tables, and fuse-pattern tables. KDA-Pilot extends this into same-ABI baseline/candidate comparison, real workloads, correctness gates, NCU evidence, and per-shape results. - Long-running optimization has started to move into Loop Engineering. The SGLang SOTA Performance Loop decomposes "chasing SOTA" into fair benchmarking, gap decision, profiling, patching, and revalidation. Humanize/RLCR adds external review, while Codex Goal can run the same loop with lower coordination overhead. - Review becomes more important. Agents can run more experiments, but they also generate more changes that look plausible and still need careful review. Developers increasingly define problems, choose evidence, design workflows, and decide whether results are ready for production paths. 2. Why SGLang Is a Good Fit for Agent-Assisted Development SGLang is a high-performance serving framework for LLMs and multimodal models. As model families and hardware paths expand, several recurring problems show up in development: - LLM paths are complex. A single performance issue may cross the Python runtime, scheduler, CUDA graph, Triton/CUDA kernels, FlashInfer/FlashAttention, distributed collectives, and model-specific wrappers. - Diffusion paths are also complex. A slower denoise pass may involve pipeline/stage partitioning, DiT blocks, attention backends, torch.compile graph breaks, CFG/SP parallelism, VAE, or custom fused kernels. - Validation is expensive. Many changes must be tested on real models and real workloads on H100, H200, B200, or RTX 5090. Local unit tests alone are not enough. - Profiles are hard to reuse manually. A single trace may contain hundreds of kernel launches. Reading Perfetto by hand can miss kernel-to-Python-source mappings and can easily mix up prefill and decode. Developers accumulate know-how while reading profiler output, such as which kernel names map to which model logic, which launch patterns suggest graph breaks, and which NCCL/attention/MLP layouts are normal. If that knowledge remains only in one person's head, the next task cannot reuse it. - Performance conclusions depend heavily on context. GPU type, shape, batch size, parallelism, precision, backend, and compile state can all change the result. An isolated microbenchmark often cannot prove real model-level benefit, so an end-to-end long-running test process is needed to repeatedly validate throughput, latency, memory, accuracy, and stability under fixed workloads. That process is both labor-intensive and time-consuming. These problems are a natural fit for agents. Launching servers, fixing workloads, collecting traces, triaging profile rows, adding tests, and recording experiment results all have clear inputs and outputs and are well suited to scripting and repeated execution. Developers need to define the boundaries: the same benchmark setup, the same profile interpretation rules, the same accuracy gates, and the conditions under which the agent should stop changing code. The agent discussed here is therefore an executor constrained by engineering workflows. Repeated SGLang development procedures can be captured as skills, letting the agent handle repetitive execution, evidence collection, and state tracking. Developers remain responsible for defining goals, judging evidence, and reviewing whether a change belongs in the real serving path. 3. From Prompt Engineering to SKILL: Protocols and Examples In the SGLang framework, a useful skill should at least answer the following questions: | Question | What the skill should capture | |---|---| | When to use it | Trigger scenarios, supported models, supported hardware, and hard-stop cases | | How to start | Preflight checks, environment variables, repository state, dependency checks, and model configuration | | How to validate | Benchmark commands, profile commands, test entry points, and accuracy gates | | How to decide | Output tables, failure modes, priorities, risk categories, and fallback conditions | | How to deliver | Artifact directories, result schemas, PR descriptions, reproduction commands, and review requirements | SGLang agent-related skills cover different layers. Some are close to source changes, such as debugging, testing, adding diffusion models, and benchmark/profile workflows. Others target cross-framework benchmarking, capacity planning, compute simulation, production incident triage, PR optimization knowledge, SGLang human-style review, and higher-level workflows such as Humanize/RLCR. 3.1 Current Skill Stack The commonly used SGLang agent-related skills fall into the following groups. | Layer | Representative skill / project | Problem it solves | |---|---|---| | CUDA crash | debug-cuda-crash | llm-serving-auto-benchmark llm-serving-capacity-planner llm-torch-profiler-analysis llm-pipeline-analysis model-compute-simulation sglang-diffusion-benchmark-profile sglang-diffusion-add-model sglang-diffusion-performance torch.compile , warmup, SP/CFG parallelism, offload, attention backend, and quantization sglang-prod-incident-triage and https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS/tree/main/skills/sglang-humanize-review sglang-humanize-review model-pr-history-knowledge sglang-sota-humanize-loop These entries turn easy-to-miss steps into executable protocols so the workflow can run, resume, and be reviewed. 3.2 Recent Optimization and Workflow Examples The following examples come from recently merged SGLang PRs. The table focuses on the full engineering path: benchmarking, profiling, localization, code changes, tests, and revalidation. | Case | Result | Key point | |---|---|---| | Router long-context tokenization deduplication, | 29% / 41% ; under 60k-token load, TTFT dropped by 34%–49% input ids fallback, and proxy body construction together, avoiding duplicate tokenization in the router and engine SGLang PR 22664 https://github.com/sgl-project/sglang/pull/22664 5.49 req/s to 9.41 req/s , about +71.4% ; mean TTFT dropped from 456.24 ms to 167.54 ms SGLang PR 27401 https://github.com/sgl-project/sglang/pull/27401 CohereLabs/command-a-plus-05-2026-w4a4 on 1x B300, request throughput improved over the previous SGLang default by +26% on chat and +21% on summarization, and beat another open-source inference framework in that setup by +4.1% / +6.8% flashinfer trtllm NVFP4 fused-MoE kernel could be used correctly in the real model path, with GSM8K/MMLU checks SGLang PR 27488 https://github.com/sgl-project/sglang/pull/27488 moonshotai/Kimi-Linear-48B-A3B-Instruct , Delta Attention prefill on B200 became 1.08x–1.52x faster than Triton; GSM8K moved from 0.915 to 0.920 , with a new regression test for realistic gate magnitudes SGLang PR 27524 https://github.com/sgl-project/sglang/pull/27524 1.63x , 1.77x , 2.07x , 2.32x , and 1.6x respectively in the reported RTX A6000 setup SGLang PR 27431 https://github.com/sgl-project/sglang/pull/27431 5.41 s to 3.84 s , about 1.41x ; peak reserved memory dropped from 71.81 GiB to 62.12 GiB , saving about 9.7 GiB In these examples, the agent mainly contributes by executing the workflow: running benchmarks, reading profiles, locating Python source, changing code, adding tests, revalidating, and preparing PR descriptions. Without skills, many steps rely on manual reminders. Once encoded as skills, the workflow becomes much easier to repeat. 4. Profiling, Review, and Loop Engineering A common mistake in SGLang performance work is to look only at total runtime, or to open Perfetto for a few minutes and decide by intuition that something "should be fused." This is even riskier for agents, because they can easily mistake a visually hot kernel for the real bottleneck. In practice, two profiler skills are usually used together. llm-torch-profiler-analysis handles the first layer of trace triage and turns a global profile into three fixed tables: Kernel Table : summarizes GPU time share, launch count, and kernel category by stage, and maps kernels back to Python source and CPU ops when possible. Overlap Opportunity Table : uses exclusive/hidden time share, dependency risk, and kernel category to identify remaining overlap or headroom. Fuse Pattern Table : compares the trace against a source-backed pattern catalog of fusion/overlap paths in SGLang, other open-source inference frameworks, and kernel libraries. These tables answer the first set of questions: which stage and which kernel take how much GPU time, which Python line they map to, and whether there is an existing fuse/overlap path to learn from. If SGLang trails another inference framework, the profiler table should explain the gap before any code change starts. The next step is llm-pipeline-analysis . Once global hotspots are known, we still need to know which forward pass, layer type, and kernel flow they belong to. This skill reads Chrome trace JSON and the model config.json , uses layer-boundary anchor kernels to split the trace into forward passes and layers, and then produces several tables for deeper analysis: Forward pass summary : separates cold-start from steady-state so warmup does not become the optimization target. Per-layer timeline : reports wall time, sum duration, and the share of categories such as MLA, MoE, GEMM, NCCL, MHC, and Hadamard for each layer. Layer cluster statistics : especially useful for models with alternating layer structures, such as NSA/hybrid-attention models with compress ratios , where C4 LIGHT, C128 HEAVY, HASH, or other layer types may dominate latency. Compute flow table : expands representative layers into concrete kernel flows with hotness, relative timestamps, and input dimensions, making it easy to jump back into Perfetto. Profile analysis therefore becomes a two-step process. First, llm-torch-profiler-analysis identifies the main conflict in the full trace. Then, llm-pipeline-analysis grounds the problem in steady-state forward passes, representative layers, and concrete kernel flows. The first step avoids choosing a direction by intuition. The second avoids staring at one global hot kernel while missing layer-type differences in the model structure. 4.1 Humanize/RLCR: Adding External Review to the Loop Humanize addresses state and review in long-running tasks. A high-risk SGLang performance task usually does not finish in one implementation pass. It may go through many rounds of benchmarking, profiling, patching, reverting, changing direction, and validating again. Humanize splits this process into two stages: - Run gen-plan first. humanize-gen-plan turns a draft requirement into a structured plan.md containing the goal description, acceptance criteria, positive/negative tests, path boundaries, milestones, and implementation notes. - Run the RLCR loop next. humanize-rlcr starts the loop from plan.md . In each round, Claude Code reads .humanize/rlcr/