Inference Products
Shared Endpoints
Access frontier models via an API
Dedicated Endpoints
Reserved GPUs. Pay per minute.
Custom Models
Your models. Our optimized stack.
Deployment Options
Our Cloud
Fully managed, pay by usage
Your Cloud
Run in your VPC. Keep your data.
Pricing
Flexible plans for any use case
Solutions
Text to audio
Image generation
Code generation
Video generation
Agentic
Case studies
Model Library
Open source models. Deploy in seconds.
DeepSeek V4.1 Flash 1017
FLUX.2 Klein 4B
GLM 5.3 Flash
MiniMax M3.1 Flash
MiniMax M3
GLM 5.3
Your own model
View all models
Day 0 access toopen frontier models.
Dedicated support foryour own model.
LLM Inference Handbook
Learn how inference works
MAX
LLM Book
Build a modern LLM from scratch
Pricing
Flexible plans for any use case
Mojo
Quest
Learn Mojo with guided challenges
Mojo
GPU Puzzles
Hands-on GPU code in Mojo
MAX
GenAI native modeling & serving
Mojo
The systems language for the AI era
Modular Alliance Program
LLVM, but for AI
Self-Hosted
Run MAX & Mojo on your hardware
Agent Skills
Our official skills for coding agents
Community
Build the future of AI together
Modular Cloud
Deploy GenAI models, our cloud or yours
MAX
GenAI native modeling & serving
Mojo
The systems language for the AI era
About
Build AI for anyone, anywhere
Careers
👋 We’re currently hiring!
Culture
What we believe
Contact Us
Request a demo
November 19, 2026
Kathy Wu
Alister Burt
Engineering
MAX is built for production inference at maximum speed. We recently redesigned our Module API to help developers build fast models even faster. Module is a standard building block in machine learning frameworks, and this post explains how and why we rebuilt this API, diving into a few key aspects:
Module
Module
Module
Module
Module
Modular has focused on production inference with state-of-the-art performance. MAX's advantage is its compilation stack: cross-architecture, optimized Mojo kernels, plus graph compilation and execution that aggressively fuse kernels and plan memory.
The legacy Module was built directly on our graph stack. The graph-forward approach delivered excellent performance, but the developer experience left room for improvement. We closed the gap by introducing a new Eager layer on top of the graph stack, then rebuilding Module on top of it.
Module
Module
If you've written a PyTorch model, the new Module should feel familiar. Here is the same layer in MAX and in PyTorch:
Module
from max.nn import Module
from max.tensor import Tensor
class MyLayer(Module):
def __init__(self, in_dim, out_dim):
self.weight = Tensor.zeros([out_dim, in_dim])
self.bias = Tensor.zeros([out_dim])
def forward(self, x):
return x @ self.weight.T + self.bias
python
import torch
from torch import nn
class MyLayer(nn.Module):
def __init__(self, in_dim, out_dim):
super().__init__()
self.weight = nn.Parameter(torch.zeros(out_dim, in_dim))
self.bias = nn.Parameter(torch.zeros(out_dim))
def forward(self, x):
return x @ self.weight.T + self.bias
The differences are intentional simplifications of the design: no super().init() call, and plain Tensors instead of nn.Parameter.
super().init()
Tensor
nn.Parameter
Agents have fundamentally changed the development cycle. While model performance remains critical for production workloads, enabling fast iteration during development is arguably more important than ever. Paying a compilation cost on every unit test and debug log is painful.
Eager execution avoids expensive graph compilation and instead starts computing results immediately. When you first run Gemma 4 using the new Module your first token appears much more quickly.
Module
Debugging models also gets a lot faster!
Module
The redesigned Module still serves our primary use case, production inference. The built-in compile method traces the model and compiles it as a graph, which lets MAX optimize the computation using knowledge about the whole model. Our graph compiler performs:
Module
compile
Module directly supports both eager and compiled execution, so model authors get the best of both worlds:
Module
model = DeepSeekV3(config)
model.load_state_dict(state_dict)
model(inputs)
compiled = model.compile(*input_types, weights=state_dict)
compiled(inputs)
The redesign was a chance to rethink how we write distributed models. The framework's role is to structure the scaling of a computation from a single GPU to many.
There is no single right way to do that. Many parallelism strategies exist because GPUs differ in compute and memory limits, models differ in layers and dimensions, and inputs differ in shape. Here are several ways to distribute a model:
Lower-level abstractions in MAX can now express data parallelism, pipeline parallelism, or tensor parallelism. A Tensor is still a logical N-D array, but it can now live on more than one device. Operations on distributed tensors behave according to specific sharding rules.
Tensor
A future post will dive deeper into the specifics of our distributed computation support. For now, see below how a row parallel linear layer is implemented:
A Module is now a Tree, a new API in MAX inspired by JAX’s Pytrees. Trees are made of Modules and Tensors which simplifies two things:
Module
Tree
Tree
Module
Tensor
Module.parameters
Module.descendants
Module
Tensor
Module.trace
Module.compile
tree.map
Tensor
These APIs also indirectly help with serialization: tree flatten produces both the flattened leaves and a TreeDef object that stores the tree structure.
flatten
TreeDef
As an example, we use Tree.map to quantize only the expert layers in a mixture-of-experts model.
Tree.map
We also removed abstractions and boilerplate that were no longer needed.
No symbolic Weight class. Weight existed to track and create weights in the graph. Now you use Tensor directly in a Module.
Weight
Weight
Tensor
Module
Default dtype and device. The layer at the top of this post defines neither. A default dtype and device context manager means a model that uses one device or dtype everywhere doesn't need to pass them to every layer. You can still set them per layer when needed.
dtype
device
dtype
device
Built-in compile. Graph construction boilerplate is gone. Every Module now has a compile method, covered in the production inference section above.
compile
Module
compile
The legacy Module implementation now lives in max.nn.legacy and will be removed in Q1 2027, so now is a good time to migrate your existing models. We’re also restructuring max.nn around the new max.tensor and max.functional libraries.
Module
max.nn.legacy
max.nn
max.tensor
max.functional libraries.
The best place to get started is the Model Bringup Workflow tutorial, which walks through building a model end to end with the new APIs.
Looking ahead, 2027 brings quantization, training, and multi-node support. Next in this series is a framework overview showing where MAX fits in the ML framework landscape, followed by posts on eager execution, distributed tensors, serialization, and more.
Modular and Qualcomm: Same code, new silicon
August 18, 2026
Why LLM Inference Needs a New Kind of Router - Part 3
June 5, 2026
Three trends from MLSys 2026
May 29, 2026
Build the future of AI with Modular
Sign up today
Signup to our Cloud Platform today to get started easily.
Browse open models
Browse our model catalog, or deploy your own custom model
Get all our latest news, announcements and updates delivered directly to your inbox. Unsubscribe at anytime.
⚠️ This form requires JavaScript to function. Please enable JavaScript in your browser to continue.
Your data is handled in accordance with Modular's Privacy Policy.
Thanks for signing up to our newsletter! 🚀
Thank you,
Modular Sales Team