{"slug": "moe-vs-edge-ai-they-are-not-the-same-thing", "title": "MoE vs Edge AI: They Are Not the Same Thing", "summary": "A developer behind the Edge0 project demonstrated a 35B-parameter language model running on an iPhone with only 1–2.5 GB of peak memory by using Mixture of Experts (MoE) routing to load only the needed expert weights from storage into RAM. The writeup distinguishes MoE, a sparse-activation architecture that selects a subset of experts per token, from edge computing, noting the two are related but solve different problems. It explains that MoE models carry large total parameter counts while activating only a fraction per token, which is what makes the on-device demo feasible.", "body_md": "Recently, I came across an [X demo](https://x.com/SamuelZengML/status/2097861839287927139) of a 35B language model running on an iPhone using only 1–2.5 GB of peak memory.\n\nThat sounds strange at first.\n\nA 35B model sounds like something that should need a lot of RAM and a powerful GPU.\n\nThe project behind the demo, [Edge0](https://github.com/Edge0-AI/edge0/), uses a technique called Mixture of Experts (MoE) to make this possible.\n\nThis also made me realise that two terms can easily get mixed up:\n\nMoE (Mixture of Experts)\n\nEdge computing / Edge AI\n\nThey are related, but they solve different problems.\n\nThis article explains both in simple terms.\n\nFirst: What is MoE?\n\nMoE stands for Mixture of Experts.\n\nAn MoE model is a neural network that contains multiple smaller networks called experts.\n\nInstead of using every expert for every input, a router decides which experts should handle the current input.\n\nFor example:\n\n```\n                Input\n                  |\n                  v\n              +-------+\n              | Router|\n              +---+---+\n                  |\n          +-------+-------+\n          |               |\n          v               v\n      Expert 3         Expert 8\n          |               |\n          +-------+-------+\n                  |\n                  v\n                Output\n```\n\nThe important part is:\n\nNot every expert is used for every token.\n\nThis is called sparse activation.\n\nAn office analogy\n\nImagine a company with 20 employees.\n\nThere are:\n\n5 software engineers\n\n3 accountants\n\n2 lawyers\n\n4 designers\n\n3 researchers\n\n3 sales people\n\nNow imagine a customer walks into the office and asks:\n\n\"Can you help me with my tax return?\"\n\nYou don't call all 20 employees into a meeting.\n\nYou send the customer to the accountants.\n\nThe accountants do the work and the rest of the employees continue doing their own jobs.\n\nThe receptionist is the router.\n\nThe employees are the experts.\n\nCustomer\n\n   |\n\n   v\n\nReceptionist / Router\n\n   |\n\n   +----> Accountant 1\n\n   |\n\n   +----> Accountant 2\n\n   |\n\n   +----> Accountant 3\n\nThat is roughly the idea behind MoE.\n\nThe model has many experts, but only some are activated for a particular token.\n\nWhy do this?\n\nBecause it lets us build models with a very large total number of parameters without having to use all of those parameters for every token.\n\nSuppose we have:\n\n8 experts\n\nx\n\n56B parameters\n\nThe model could have around 56B total parameters.\n\nBut perhaps the router only activates 2 experts for a particular token.\n\nSo the amount of expert computation is much smaller than running all 56B parameters every time.\n\nThis gives us a useful distinction:\n\nTotal parameters\n\nHow many parameters exist in the whole model.\n\nActive parameters\n\nHow many parameters are actually being used for a particular token.\n\nYou might therefore see an MoE model described with something like:\n\n47B total parameters\n\n13B active parameters\n\nThe model has a large total capacity, but only part of it is used at once.\n\nBut there is a problem\n\nIf you have a huge number of experts, those experts still have weights.\n\nImagine:\n\nModel\n\n |\n\n +-- Expert 1\n\n +-- Expert 2\n\n +-- Expert 3\n\n +-- Expert 4\n\n +-- ...\n\n +-- Expert 32\n\nEven if we only need Expert 3 right now, the other experts still exist.\n\nSo where do all those weights live?\n\nThis is where things get interesting.\n\nEdge0 and the 35B iPhone example\n\nThe Edge0 demo I mentioned earlier shows a 35B language model running on an iPhone with around 1–2.5 GB of peak memory.\n\nThe important idea is that the entire model does not need to sit in RAM at the same time.\n\nThink of the iPhone like this:\n\n```\n         iPhone storage\n    +----------------------+\n    | Expert 1             |\n    | Expert 2             |\n    | Expert 3             |\n    | Expert 4             |\n    | ...                  |\n    | Expert N             |\n    +----------+-----------+\n               |\n               | load what is needed\n               v\n             RAM\n      +----------------+\n      | Active experts |\n      +-------+--------+\n              |\n              v\n          Compute\n```\n\nThe model can keep its large collection of weights in storage and bring the required pieces into memory.\n\nThe router helps determine which experts are needed.\n\nThis is one of the reasons MoE is useful for systems like Edge0.\n\nSo is Edge0 \"edge computing\"?\n\nNot exactly.\n\nThis is where the terminology becomes confusing.\n\nMoE describes the architecture of the AI model.\n\nEdge computing describes where/how computation happens.\n\nThey are different dimensions.\n\nYou can have:\n\n```\n         AI Model\n            |\n    +-------+-------+\n    |               |\n   Dense            MoE\n    |               |\n    +-------+-------+\n            |\n    Where does it run?\n            |\n   +--------+--------+\n   |                 |\n  Cloud             Edge\n```\n\nAn MoE model can run in the cloud.\n\nAn MoE model can also run on a phone.\n\nAnd an edge device can run a dense model.\n\nThere is no requirement that edge AI must use MoE.\n\nThen what is Edge Computing?\n\nThe basic idea behind edge computing is:\n\nMove computation closer to where the data is being produced or where the user is.\n\nInstead of always doing:\n\nPhone\n\n  |\n\n  | Internet\n\n  v\n\nCloud server\n\n  |\n\n  v\n\nAI model\n\n  |\n\n  v\n\nResponse\n\n  |\n\n  v\n\nPhone\n\nyou can do:\n\nPhone\n\n  |\n\n  v\n\nAI model\n\n  |\n\n  v\n\nResponse\n\nThe computation happens on the device itself, or on infrastructure close to the device.\n\nFor AI, this is often called Edge AI or on-device AI.\n\nCactus is a good example\n\nA project like Cactus Compute is much closer to what I mean by an edge AI platform.\n\nThe goal is not primarily:\n\n\"Let's create a new type of neural network.\"\n\nInstead, the goal is closer to:\n\n\"How do we make AI inference work efficiently on devices such as phones and other edge hardware?\"\n\nThat involves things like:\n\nmodel quantisation\n\nhardware acceleration\n\nefficient kernels\n\nmemory management\n\nlocal inference\n\ndevice SDKs\n\ndeciding when computation should happen locally or remotely\n\nFor example, a mobile application could have:\n\n```\n             Mobile App\n                 |\n                 v\n              Cactus\n                 |\n      +----------+----------+\n      |                     |\n      v                     v\nRun locally             Use cloud\n      |                     |\n      v                     v\n   Phone                Server\n```\n\nA simple request could run locally.\n\nA request that requires a larger model could be sent to a server.\n\nThe exact architecture depends on the application.\n\nThe difference in one sentence\n\nIf you remember nothing else from this article, remember this:\n\nMoE is about how an AI model is structured. Edge computing is about where the computation happens.\n\nThat's the core difference.\n\nAnother analogy\n\nLet's go back to our office.\n\nMoE\n\nMoE is about how the company is organised.\n\nYou have many specialists and a receptionist decides who should handle each customer.\n\nCustomer\n\n   |\n\n   v\n\nReceptionist\n\n   |\n\n   +--> Lawyer\n\n   |\n\n   +--> Accountant\n\n   |\n\n   +--> Engineer\n\nThat's the model architecture.\n\nEdge computing\n\nEdge computing is about where the office is located.\n\nInstead of having one giant office in London:\n\n```\n                 Customer\n                     |\n                     v\n              London Office\n```\n\nyou could have smaller offices near your customers:\n\n```\n    Customer                Customer\n       |                       |\n       v                       v\n Local Office             Local Office\n```\n\nThe work happens closer to the customer.\n\nThat's the location of computation.\n\nThey can also be combined\n\nThis is the interesting part.\n\nYou could have:\n\nAn MoE model running on an edge device.\n\n```\n         iPhone\n    +----------------+\n    |                |\n    |    MoE Model   |\n    |                |\n    |    Router      |\n    |       |        |\n    |   +---+---+    |\n    |   |       |    |\n    | Expert A Expert B\n    |                |\n    +----------------+\n```\n\nThis is basically the kind of problem Edge0 is interested in.\n\nYou have:\n\n→ only some experts need to be active.\n\nEdge\n\n→ the model runs on the user's device.\n\nTogether, you can potentially run much larger models on hardware with limited memory.\n\nBut edge does not require MoE\n\nYou could just as easily run a normal dense model on an iPhone:\n\niPhone\n\n   |\n\n   v\n\nDense LLM\n\n   |\n\n   v\n\nResponse\n\nThere is no router.\n\nThere are no experts.\n\nIt's still edge AI because the inference happens on the device.\n\nAnd MoE does not require edge computing\n\nYou can also run an MoE model on a huge server:\n\n```\n              Cloud GPU cluster\n                     |\n                     v\n                 MoE model\n                     |\n          +----------+----------+\n          |                     |\n       Expert A              Expert F\n```\n\nThe model is still MoE.\n\nIt doesn't become \"edge\" simply because it uses experts.\n\nWhy does this matter?\n\nBecause when you hear:\n\n\"35B model running on an iPhone\"\n\nyou might immediately think:\n\n\"They somehow squeezed a 35B model into an iPhone's RAM.\"\n\nThat's not quite the right way to think about it.\n\nA better mental model is:\n\n```\n         35B total model\n                |\n                v\n             Router\n                |\n        \"What do I need?\"\n                |\n      +---------+---------+\n      |                   |\n      v                   v\n  Expert 4             Expert 17\n      |                   |\n      +---------+---------+\n                |\n                v\n             Output\n```\n\nThe system is taking advantage of the fact that the entire model does not have to be actively used at once.\n\nThen Edge0 adds another layer of systems engineering around this.\n\nIt tries to manage where the model's weights live and when they are brought into memory.", "url": "https://wpnews.pro/news/moe-vs-edge-ai-they-are-not-the-same-thing", "canonical_source": "https://dev.to/ceasermikes002/moe-vs-edge-ai-they-are-not-the-same-thing-1pc6", "published_at": "2026-09-12 09:49:17+00:00", "updated_at": "2026-09-12 10:01:20.713982+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research", "ai-infrastructure", "ai-tools"], "entities": ["Edge0", "iPhone", "SamuelZengML"], "alternates": {"html": "https://wpnews.pro/news/moe-vs-edge-ai-they-are-not-the-same-thing", "markdown": "https://wpnews.pro/news/moe-vs-edge-ai-they-are-not-the-same-thing.md", "text": "https://wpnews.pro/news/moe-vs-edge-ai-they-are-not-the-same-thing.txt", "jsonld": "https://wpnews.pro/news/moe-vs-edge-ai-they-are-not-the-same-thing.jsonld"}}