MoE Analysis Qwen[3.5|3.6]35B-A3B

An analysis of two Mixture-of-Experts models, Qwen 3.5 35B-A3B and Qwen 3.6 35B-A3B, captured per-token router statistics across every MoE layer using first-turn MT-Bench prompts, recording which experts the router selected and how confident each choice was. The study, published by G.S. Laller, reports expert-selection and router-confidence data by model, color and layer.

/MoE routing ./ G.S. LALLER /Notes MoE Analysis, which experts fire, and how sure is the router? We took the MT-Bench prompts first turn only , ran inference on each of them with two Mixture-of-Experts models, and captured the statistics of every MoE layer: for each token, which experts the router picked, and how confident that choice was. Model Colour by Layer Loading data/manifest.json …