# MoE vs Edge AI: They Are Not the Same Thing > Source: > Published: 2026-09-12 09:49:17+00:00 Recently, I came across an [X demo](https://x.com/SamuelZengML/status/2097861839287927139) of a 35B language model running on an iPhone using only 1–2.5 GB of peak memory. That sounds strange at first. A 35B model sounds like something that should need a lot of RAM and a powerful GPU. The project behind the demo, [Edge0](https://github.com/Edge0-AI/edge0/), uses a technique called Mixture of Experts (MoE) to make this possible. This also made me realise that two terms can easily get mixed up: MoE (Mixture of Experts) Edge computing / Edge AI They are related, but they solve different problems. This article explains both in simple terms. First: What is MoE? MoE stands for Mixture of Experts. An MoE model is a neural network that contains multiple smaller networks called experts. Instead of using every expert for every input, a router decides which experts should handle the current input. For example: ``` Input | v +-------+ | Router| +---+---+ | +-------+-------+ | | v v Expert 3 Expert 8 | | +-------+-------+ | v Output ``` The important part is: Not every expert is used for every token. This is called sparse activation. An office analogy Imagine a company with 20 employees. There are: 5 software engineers 3 accountants 2 lawyers 4 designers 3 researchers 3 sales people Now imagine a customer walks into the office and asks: "Can you help me with my tax return?" You don't call all 20 employees into a meeting. You send the customer to the accountants. The accountants do the work and the rest of the employees continue doing their own jobs. The receptionist is the router. The employees are the experts. Customer | v Receptionist / Router | +----> Accountant 1 | +----> Accountant 2 | +----> Accountant 3 That is roughly the idea behind MoE. The model has many experts, but only some are activated for a particular token. Why do this? Because it lets us build models with a very large total number of parameters without having to use all of those parameters for every token. Suppose we have: 8 experts x 56B parameters The model could have around 56B total parameters. But perhaps the router only activates 2 experts for a particular token. So the amount of expert computation is much smaller than running all 56B parameters every time. This gives us a useful distinction: Total parameters How many parameters exist in the whole model. Active parameters How many parameters are actually being used for a particular token. You might therefore see an MoE model described with something like: 47B total parameters 13B active parameters The model has a large total capacity, but only part of it is used at once. But there is a problem If you have a huge number of experts, those experts still have weights. Imagine: Model | +-- Expert 1 +-- Expert 2 +-- Expert 3 +-- Expert 4 +-- ... +-- Expert 32 Even if we only need Expert 3 right now, the other experts still exist. So where do all those weights live? This is where things get interesting. Edge0 and the 35B iPhone example The Edge0 demo I mentioned earlier shows a 35B language model running on an iPhone with around 1–2.5 GB of peak memory. The important idea is that the entire model does not need to sit in RAM at the same time. Think of the iPhone like this: ``` iPhone storage +----------------------+ | Expert 1 | | Expert 2 | | Expert 3 | | Expert 4 | | ... | | Expert N | +----------+-----------+ | | load what is needed v RAM +----------------+ | Active experts | +-------+--------+ | v Compute ``` The model can keep its large collection of weights in storage and bring the required pieces into memory. The router helps determine which experts are needed. This is one of the reasons MoE is useful for systems like Edge0. So is Edge0 "edge computing"? Not exactly. This is where the terminology becomes confusing. MoE describes the architecture of the AI model. Edge computing describes where/how computation happens. They are different dimensions. You can have: ``` AI Model | +-------+-------+ | | Dense MoE | | +-------+-------+ | Where does it run? | +--------+--------+ | | Cloud Edge ``` An MoE model can run in the cloud. An MoE model can also run on a phone. And an edge device can run a dense model. There is no requirement that edge AI must use MoE. Then what is Edge Computing? The basic idea behind edge computing is: Move computation closer to where the data is being produced or where the user is. Instead of always doing: Phone | | Internet v Cloud server | v AI model | v Response | v Phone you can do: Phone | v AI model | v Response The computation happens on the device itself, or on infrastructure close to the device. For AI, this is often called Edge AI or on-device AI. Cactus is a good example A project like Cactus Compute is much closer to what I mean by an edge AI platform. The goal is not primarily: "Let's create a new type of neural network." Instead, the goal is closer to: "How do we make AI inference work efficiently on devices such as phones and other edge hardware?" That involves things like: model quantisation hardware acceleration efficient kernels memory management local inference device SDKs deciding when computation should happen locally or remotely For example, a mobile application could have: ``` Mobile App | v Cactus | +----------+----------+ | | v v Run locally Use cloud | | v v Phone Server ``` A simple request could run locally. A request that requires a larger model could be sent to a server. The exact architecture depends on the application. The difference in one sentence If you remember nothing else from this article, remember this: MoE is about how an AI model is structured. Edge computing is about where the computation happens. That's the core difference. Another analogy Let's go back to our office. MoE MoE is about how the company is organised. You have many specialists and a receptionist decides who should handle each customer. Customer | v Receptionist | +--> Lawyer | +--> Accountant | +--> Engineer That's the model architecture. Edge computing Edge computing is about where the office is located. Instead of having one giant office in London: ``` Customer | v London Office ``` you could have smaller offices near your customers: ``` Customer Customer | | v v Local Office Local Office ``` The work happens closer to the customer. That's the location of computation. They can also be combined This is the interesting part. You could have: An MoE model running on an edge device. ``` iPhone +----------------+ | | | MoE Model | | | | Router | | | | | +---+---+ | | | | | | Expert A Expert B | | +----------------+ ``` This is basically the kind of problem Edge0 is interested in. You have: → only some experts need to be active. Edge → the model runs on the user's device. Together, you can potentially run much larger models on hardware with limited memory. But edge does not require MoE You could just as easily run a normal dense model on an iPhone: iPhone | v Dense LLM | v Response There is no router. There are no experts. It's still edge AI because the inference happens on the device. And MoE does not require edge computing You can also run an MoE model on a huge server: ``` Cloud GPU cluster | v MoE model | +----------+----------+ | | Expert A Expert F ``` The model is still MoE. It doesn't become "edge" simply because it uses experts. Why does this matter? Because when you hear: "35B model running on an iPhone" you might immediately think: "They somehow squeezed a 35B model into an iPhone's RAM." That's not quite the right way to think about it. A better mental model is: ``` 35B total model | v Router | "What do I need?" | +---------+---------+ | | v v Expert 4 Expert 17 | | +---------+---------+ | v Output ``` The system is taking advantage of the fact that the entire model does not have to be actively used at once. Then Edge0 adds another layer of systems engineering around this. It tries to manage where the model's weights live and when they are brought into memory.