FreeToken Explained: Run 290B+ MoE Models on One Gaming GPU
FreeToken, a local inference tool, enables running mixture-of-experts models with over 290 billion parameters on a single consumer GPU by streaming only active experts from system RAM, avoiding the need for multi-GPU set…