Agnes-3.0-Flash Preview: Specs and Benchmarks of the Open Model Agnes AI released Agnes-3.0-Flash Preview, a 33-billion-parameter open-weight multimodal language model under an Apache 2.0 license with a 262,144-token context window and a hybrid attention architecture. Of the model's 72 decoder layers, 54 use a gated delta rule with fixed-size recurrent state and 18 use full global attention in a repeating 3:1 pattern, which limits KV cache growth; the bf16 checkpoint weighs roughly 66GB and Agnes AI recommends a single H200 (141GB) or H100 (80GB) GPU for deployment. The Hugging Face release has drawn over 200 likes and more than 1,000 downloads, and Agnes AI notes the open checkpoint differs from the production API version of Agnes 3.0 Flash, which uses a 1M-token context window and a different configuration. Agnes-3.0-Flash Preview: Specs and Benchmarks of the Open Model Agnes-3.0-Flash Preview is a 33B open-weight multimodal model with 262K context, hybrid attention, and tool calling. Full specs inside. What is Agnes-3.0-Flash Preview? Agnes-3.0-Flash Preview is an open-weight, multimodal language model released by Agnes AI under an Apache 2.0 license. It has 33 billion parameters, a 262,144-token context window, and a hybrid attention architecture that mixes recurrent state layers with standard global attention. It handles text, images, and video, supports tool calling, and lets developers dial reasoning effort up or down. The weights are published on Hugging Face, where the release has drawn over 200 likes and more than 1,000 downloads. TL;DR - Open-weight release Agnes-3.0-Flash Preview ships as a downloadable 33B-parameter checkpoint on Hugging Face under Apache 2.0, separate from the larger production API model of the same family name. - Long context window The model supports up to 262,144 tokens, achieved partly through a hybrid attention design that limits how much KV cache grows as input length increases. - Mixed layer architecture Of 72 decoder layers, 54 use a gated delta rule a recurrent mechanism with fixed-size state and only 18 use full global attention, in a repeating 3:1 pattern. - Multimodal by default The model ingests text, images, and video through a bundled vision tower and processor, and it exposes adjustable reasoning levels high, medium, low, or off via its chat template. - Competitive benchmark showing Across IFBench, SciCode, GPQA Diamond, AA-LCR, and AA-Omniscience, the Preview checkpoint lands in the middle of the pack against similarly sized open models, trailing larger mixture-of-experts systems on some tests while beating them on others. - Single-GPU deployable Agnes AI recommends one H200 141GB or H100 80GB GPU at bf16 precision, with the bf16 checkpoint weighing in around 66GB on disk. - Important version distinction This open checkpoint is not the same as the production/API version of Agnes 3.0 Flash listed on Artificial Analysis, which uses a different configuration and a 1M-token context window; benchmark results between the two should not be conflated. - ✕a coding agent - ✕no-code - ✕vibe coding - ✕a faster Cursor The one that tells the coding agents what to build. How does the hybrid attention architecture work? Agnes-3.0-Flash Preview’s core design choice is splitting its 72 decoder layers into two types. Three out of every four layers run a gated delta rule, a recurrent mechanism where each layer carries a fixed-size internal state rather than a cache that grows with sequence length. The remaining layer in each group of four uses standard global attention with a full key-value cache. In practice, that means only 18 of the 72 layers actually accumulate a KV cache as context grows. The other 54 layers process tokens through per-layer state that stays constant in size no matter how long the input gets. This is the architectural trick behind supporting a 262,144-token context window without the memory cost scaling the way it would in a pure transformer. The global attention layers use grouped-query attention with 24 query heads and 4 key-value heads a 6:1 ratio , a head dimension of 256, and RMS normalization applied to queries and keys before a sigmoid-gated output. The delta-rule layers use 16 key heads and 48 value heads at a head dimension of 128, preceded by a causal convolution kernel size 4 and followed by gated RMS normalization. The recurrent state itself is kept in fp32 for numerical stability. Position information comes from a three-axis rotary embedding scheme covering text, height, and width, useful for interleaving text and visual tokens. Each layer also runs a SwiGLU feed-forward block with an intermediate size of 17,408, plus a smaller parallel SwiGLU branch of 2,048 in every layer. The vocabulary sits at 248,320 tokens, and the vision tower is a 27-layer network with hidden size 1,152 that patches images at 16x16, merges them 2x2, and projects the result into the model’s 5,120-dimensional hidden space. What do the benchmark numbers actually show? Agnes AI published reference scores across five benchmarks: IFBench instruction following , SciCode scientific coding , GPQA Diamond graduate-level science Q&A , AA-LCR long-context reasoning , and AA-Omniscience Accuracy broad knowledge recall . These were compared against a spread of contemporary open and closed models ranging from 27B dense models to trillion-parameter mixture-of-experts systems like Kimi K2.5 and MiniMax M3. The results are mixed rather than uniformly dominant, which is typical for a 33B model going up against systems with far more total or active parameters. On IFBench, Agnes-3.0-Flash Preview scored 74.20, ahead of Qwen3.6-35B-A3B 64.4 and Kimi K2.5 43.7 , but behind larger models like Qwen3.8 Flash Next 81.3 and MiniMax M3 82.9 . On GPQA Diamond, it posted 85.05, competitive with Qwen3.6-35B-A3B 84.1 and Muse Glimmer 83.5 , but well behind DeepSeek V4 Flash 0731 90.8 and MiniMax M3 92.9 . Where it stands out is AA-Omniscience Accuracy, where it scored 23.00, edging out several larger models including Qwen3.8 18.4 and MiniMax M3 16.7 , though still far behind Gemini 3.5 Flash’s 51.4. On SciCode, it trails most of the field at 38.08, ahead only of Qwen3.6-35B-A3B 35.8 . Agnes AI’s own documentation flags that these figures come from different harnesses and model snapshots, so they function as reference points rather than a controlled head-to-head test. Parameter counts in the comparison table also mix total and active parameter figures mixture-of-experts models report both , which complicates any straightforward “size versus performance” reading. Is Agnes-3.0-Flash Preview the same as the production Agnes 3.0 Flash model? No. This is a distinction the model card goes out of its way to clarify. The Hugging Face repository holds an earlier open-weight Preview checkpoint , distinct from a separate production/API checkpoint that appears on Artificial Analysis’s model tracking site. The two differ in meaningful ways: the Preview checkpoint has a 262,144-token context window, while the production/API version supports up to 1 million tokens. They also use different configurations. The repository was originally published without the “Preview” label, which the model card now explicitly adds to avoid confusion. Any benchmark numbers attributed to the production/API model on third-party tracking sites don’t apply to the open weights covered here, and vice versa. For anyone benchmarking or citing Agnes 3.0 Flash, checking which checkpoint produced the numbers matters. What hardware and setup does it need to run? Agnes AI recommends a single NVIDIA H200 141GB or H100 80GB GPU at bf16 precision, with tensor parallelism of 1 as the default and 2 available for extending context length and concurrency. The bf16 checkpoint occupies roughly 66GB on disk, and the company suggests 128GB or more of host memory. The model requires trust remote code=True when loading through Hugging Face Transformers version 5.12 or newer , since it ships a custom model implementation rather than relying on a standard architecture class. Image and video processing go through a bundled processor that depends on torchvision. For production serving, Agnes AI provides an SGLang integration via a serve.sh script that patches three files into a stock SGLang Docker image. This exposes an OpenAI-compatible API endpoint, supports streaming, and accepts the same tools= and reasoning effort= parameters used in the direct Transformers workflow. Tool calls are emitted in a structured