{"slug": "a-few-notable-open-weight-models-this-week", "title": "A Few Notable Open-Weight Models This Week", "summary": "Open-weight model releases this week include Nanbeige 4.2 3B with looped depth sharing, poolside's Laguna S 2.1 (118B sparse MoE, 8B active, 1M-token context), Motif-3-Beta (314B-A13B sparse MoE with Grouped Differential Latent Attention), Upstage's Solar Open 2 (250B-A15B hybrid MoE), Cisco's Antares 1B for cybersecurity, and BTL-3 (LoRA adapter for Qwen3.6-27B). Sebastian Raschka highlighted these models in a Substack note, emphasizing their architectural innovations and practical utility for running AI on personal hardware.", "body_md": "# A Few Notable Open-Weight Models This Week\n\nYes, open-source / open-weight models are important for a healthy AI ecosystem. That’s how we can verify things, check claims, and keep up outside the closed labs. Plus, it gives us the freedom to run AI on our own hardware if we are not ready to share personal data and IPs with closed labs through using their models. (Not that proprietary models are bad, actually I use them a lot as well, but it wouldn’t be healthy not to have any alternatives.)\n\nAnyway, while pretty much everyone is waiting for the Kimi K3 and Ling 3.0 weights to land on the model hub any day now, there were quite a few other interesting new open-weight model releases the past week. Yes, one of those weeks!\n\nSo, here are the architecture pics along with some notes on what I found most interesting:\n\n1) Nanbeige 4.2 3B uses [looped depth sharing](/glossary/#looped-depth-sharing). This basically means it runs the same 22-layer (=transformer block) stack twice. So, it extends the 22-layer architecture to 44 layers, but without duplicating the weights. (2x the transformer block compute but same memory footprint.)\n\nWhy? The info is a bit sparse, but section 2.1 of the Nanbeige 4.2 technical report says two passes gave the best trade-off and retained about 75% of the token efficiency of a standard architecture. More passes gave barely any gains but made the training much slower and much more expensive.\n\n2) Laguna S 2.1 is poolside’s Laguna model in a really nice size: 118B sparse MoE with 8B active parameters and a 1M-token [context window](/glossary/#context-length). Otherwise, the architecture is pretty standard. It uses 36 sliding-window and 12 global (gated-)GQA layers. However, given this size, and the fact that it (just barely) runs on my DGX Spark (uses less than 80 GB of RAM), this is right now the most interesting model for me personally. It’s 3x bigger and thus a tad slower but maybe a good candidate as daily-driver-Qwen3.6-35B-replacement. (Still waiting on some more independent performance benchmarks though.)\n\n3) Motif-3-Beta is a new 314B-A13B sparse MoE that is somewhat based on DeepSeek V4 in terms of mHC and [latent attention](/glossary/#mla). But it uses a new component, Grouped Differential Latent Attention, which is inspired by Multi-head Latent Attention. I probably should write an article about this some time, but for now, the tl;dr is as follows. Regular MLA compresses the keys and values into a smaller latent representation to mainly reduce the KV cache size. GDLA does a similar low-rank compression but puts the attention heads into groups and also learns a noise head for each group where the noise gets subtracted for filtering purposes… Anyway, a topic for another day!\n\n4) Solar Open 2 is a new 250B-A15B hybrid [MoE](/glossary/#moe) by Upstage that interleaves three Kimi Delta Attention layers with one GQA layer.\n\n5) Antares 1B is a small model (and there is also an even smaller 0.3B variant) from Cisco that starts with the IBM Granite 4.0 1B backbone and uses SFT plus [GRPO](/glossary/#grpo) for terminal-based cybersecurity stuff. It is a nice example of task-specific post-training on a genuinely small model.\n\n6) BTL-3 is a rank-32 LoRA adapter for Qwen3.6-27B aimed at coding agents and structured tool use. The really strong [benchmark](/glossary/#benchmark) performance suggests that LoRA adapters are still a useful tool/technique in 2026.\n\nI added all six to the [LLM Architecture Gallery](/llm-architecture-gallery/) for some additional details.\n\nSource: lightly edited website version of my [Substack note](https://substack.com/@rasbt/note/c-302083551).\n\n## Read Next\n\n[Correction for Listing 6.5 in Build a Reasoning Model From Scratch Short correction note for the random seed in Listing 6.5 on page 198 of Build a Reasoning Model From Scratch.](/blog/2026/reasoning-model-listing-6-5-correction.html)\n\n[Inkling: A New Open-Weight 975B MoE with a Few Surprises Short note on Thinking Machines Lab's 975B Inkling model, including benchmarks, sparse MoE design, short convolutions, RMSNorm, and position bias.](/blog/2026/inkling-architecture-benchmark-notes.html)\n\n[200,000 Subscribers Short note celebrating Ahead of AI reaching 200,000 subscribers.](/blog/2026/ahead-of-ai-reached-200000-subscribers.html)", "url": "https://wpnews.pro/news/a-few-notable-open-weight-models-this-week", "canonical_source": "https://sebastianraschka.com/blog/2026/notable-open-weight-models-this-week.html", "published_at": "2026-07-26 08:47:44+00:00", "updated_at": "2026-07-26 14:27:00.906010+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-products", "ai-infrastructure"], "entities": ["Nanbeige 4.2 3B", "poolside", "Laguna S 2.1", "Motif-3-Beta", "Upstage", "Solar Open 2", "Cisco", "Antares 1B"], "alternates": {"html": "https://wpnews.pro/news/a-few-notable-open-weight-models-this-week", "markdown": "https://wpnews.pro/news/a-few-notable-open-weight-models-this-week.md", "text": "https://wpnews.pro/news/a-few-notable-open-weight-models-this-week.txt", "jsonld": "https://wpnews.pro/news/a-few-notable-open-weight-models-this-week.jsonld"}}