DigitalSpaceport ran DeepSeek V4 Flash, a 284-billion-parameter Mixture-of-Experts model, on a single used RTX 3090 (24GB) plus system RAM, achieving about 10 to 11 tokens per second for chat, but the…
Researchers from UC Berkeley and MIT released FreeToken, an open-source inference engine that enables frontier Mixture-of-Experts (MoE) models to run on consumer hardware via dynamic co-execution, ach…
FreeToken, an LLM engine designed to maximize bandwidth across all hardware, faces a fundamental challenge: bit-exact reproducibility is impossible across different CPU and GPU architectures due to di…
Researchers from UT Austin and UC Berkeley released FreeToken, an edge-native Mixture-of-Experts (MoE) serving engine that adapts model execution to available hardware bandwidth, enabling a 753B GLM-5…
FreeToken, a new serving tool, enables running frontier mixture-of-experts (MoE) models with hundreds of billions of parameters on a single consumer GPU by keeping most experts in system RAM and strea…
DeepSeek V4 Flash, a mixture-of-experts model, ran at roughly 10 to 11 tokens per second on a single RTX 3090 with 192GB of system RAM in tests by FreeToken's desktop app, while a dense Qwen 3.8 27B m…
FreeToken, a local inference tool, enables running mixture-of-experts models with over 290 billion parameters on a single consumer GPU by streaming only active experts from system RAM, avoiding the ne…
Researchers from UC Berkeley and UT Austin released FreeToken, an edge-native Mixture-of-Experts serving engine that runs the 753B-parameter GLM-5.2 model on a single workstation GPU, achieving 35B at…
FreeToken, an open-source inference engine developed by UC Berkeley researchers including Shuo Yang and Xiaoze Fan, reports decoding DeepSeek-V4-Flash, a 284B-parameter Mixture-of-Experts model, at 22…
FreeToken, a new inference engine from FlashML, lets users run frontier models on gaming GPUs at interactive speeds, with Qwen3.6 35B running on an 8GB RTX 4060 laptop at 39 tokens per second, DeepSee…
Researchers introduced FreeToken, an edge-native Mixture-of-Experts (MoE) serving system that enables frontier-scale AI models to run on personal hardware, supporting over 20 MoE models and real codin…
FlashML released FreeToken, an edge-native Mixture-of-Experts (MoE) serving engine that runs 290B+ parameter frontier MoE models locally on consumer gaming PCs at interactive speeds. The engine suppor…