{"slug": "mawforge-memory-bounded-expert-materialization-for-local-mixture-of-experts", "title": "MawForge: Memory-Bounded Expert Materialization for Local Mixture-of-Experts Inference", "summary": "Researchers introduced MawForge, a system for running sparse Mixture-of-Experts language models on memory-constrained local machines by storing the full model on disk and materializing expert tensors into a bounded cache on demand. The approach is effective as a measurement substrate but not as a cache-maximization policy, with performance depending on factors like expert reuse and memory pressure.", "body_md": "arXiv:2607.09686v1 Announce Type: new\nAbstract: Sparse Mixture-of-Experts (MoE) language models separate total parameter count from per-token active computation, but local inference systems often still require the full model, key-value cache, runtime buffers, and operatingsystem headroom to fit in fast memory. MawForge tests a different systems hypothesis: local MoE serving can be made practical on constrained unified-memory machines by storing the full model on disk, keeping common tensors resident, and materializing routed expert tensors into a bounded execution cache on demand. The central finding is that MawForge is effective as a bounded execution mechanism and measurement substrate for local MoE inference, but not as a cache-maximization policy. Performance depends on balancing expert reuse against resident footprint, KV-cache size, quantization, route locality, and macOS memory pressure.", "url": "https://wpnews.pro/news/mawforge-memory-bounded-expert-materialization-for-local-mixture-of-experts", "canonical_source": "https://arxiv.org/abs/2607.09686", "published_at": "2026-07-14 04:00:00+00:00", "updated_at": "2026-07-14 04:22:13.742899+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-infrastructure"], "entities": ["MawForge"], "alternates": {"html": "https://wpnews.pro/news/mawforge-memory-bounded-expert-materialization-for-local-mixture-of-experts", "markdown": "https://wpnews.pro/news/mawforge-memory-bounded-expert-materialization-for-local-mixture-of-experts.md", "text": "https://wpnews.pro/news/mawforge-memory-bounded-expert-materialization-for-local-mixture-of-experts.txt", "jsonld": "https://wpnews.pro/news/mawforge-memory-bounded-expert-materialization-for-local-mixture-of-experts.jsonld"}}