cd /news/artificial-intelligence/modular-reintroducing-maxs-module · home › topics › artificial-intelligence › article
[ARTICLE · art-172763] src=modular.com ↗ pub= topic=artificial-intelligence verified=true sentiment=↑ positive

Modular: Reintroducing MAX’s Module

Modular rebuilt MAX's Module API on a new Eager execution layer, letting developers run models like Gemma 4 without paying graph compilation costs on every unit test or debug log, the company said in a November 19, 2026 engineering post by Kathy Wu and Alister Burt. The redesigned Module drops the super().__init__() call and uses plain Tensors instead of nn.Parameter, making it familiar to anyone who has written a PyTorch model, while the built-in compile method preserves MAX's production inference path. Modular said the change targets faster iteration during development without sacrificing the cross-architecture Mojo kernels and graph compilation that underpin MAX's performance.

key takeaway

Rebuilding MAX's Module on an Eager layer skips graph compilation so the first token of a model like Gemma 4 appears sooner and unit tests and debug runs no longer incur compile cost, while the built-in compile method retains the production path.

read6 min views1 publishedNov 19, 2026
Modular: Reintroducing MAX’s Module
Image: Modular (auto-discovered)

Inference Products

Shared Endpoints

Access frontier models via an API

Dedicated Endpoints

Reserved GPUs. Pay per minute.

Custom Models

Your models. Our optimized stack.

Deployment Options

Our Cloud

Fully managed, pay by usage

Your Cloud

Run in your VPC. Keep your data.

Pricing

Flexible plans for any use case

Solutions

Text to audio

Image generation

Code generation

Video generation

Agentic

Case studies

Model Library

Open source models. Deploy in seconds.

DeepSeek V4.1 Flash 1017

FLUX.2 Klein 4B

GLM 5.3 Flash

MiniMax M3.1 Flash

MiniMax M3

GLM 5.3

Your own model

View all models

Day 0 access toopen frontier models.

Dedicated support foryour own model.

LLM Inference Handbook

Learn how inference works

MAX

LLM Book

Build a modern LLM from scratch

Pricing

Flexible plans for any use case

Mojo

Quest

Learn Mojo with guided challenges

Mojo

GPU Puzzles

Hands-on GPU code in Mojo

MAX

GenAI native modeling & serving

Mojo

The systems language for the AI era

Modular Alliance Program

LLVM, but for AI

Self-Hosted

Run MAX & Mojo on your hardware

Agent Skills

Our official skills for coding agents

Community

Build the future of AI together

Modular Cloud

Deploy GenAI models, our cloud or yours

MAX

GenAI native modeling & serving

Mojo

The systems language for the AI era

About

Build AI for anyone, anywhere

Careers

👋 We’re currently hiring!

Culture

What we believe

Contact Us

Request a demo

November 19, 2026

Kathy Wu

Alister Burt

Engineering

MAX is built for production inference at maximum speed. We recently redesigned our Module API to help developers build fast models even faster. Module is a standard building block in machine learning frameworks, and this post explains how and why we rebuilt this API, diving into a few key aspects:

Module

Module

Module

Module

Module

Modular has focused on production inference with state-of-the-art performance. MAX's advantage is its compilation stack: cross-architecture, optimized Mojo kernels, plus graph compilation and execution that aggressively fuse kernels and plan memory.

The legacy Module was built directly on our graph stack. The graph-forward approach delivered excellent performance, but the developer experience left room for improvement. We closed the gap by introducing a new Eager layer on top of the graph stack, then rebuilding Module on top of it.

Module

Module

If you've written a PyTorch model, the new Module should feel familiar. Here is the same layer in MAX and in PyTorch:

Module

from max.nn import Module
from max.tensor import Tensor

class MyLayer(Module):
    def __init__(self, in_dim, out_dim):
        self.weight = Tensor.zeros([out_dim, in_dim])
        self.bias = Tensor.zeros([out_dim])

    def forward(self, x):
        return x @ self.weight.T + self.bias
python
import torch
from torch import nn

class MyLayer(nn.Module):
    def __init__(self, in_dim, out_dim):
        super().__init__()
        self.weight = nn.Parameter(torch.zeros(out_dim, in_dim))
        self.bias = nn.Parameter(torch.zeros(out_dim))

    def forward(self, x):
        return x @ self.weight.T + self.bias

The differences are intentional simplifications of the design: no super().init() call, and plain Tensors instead of nn.Parameter.

super().init()

Tensor

nn.Parameter

Agents have fundamentally changed the development cycle. While model performance remains critical for production workloads, enabling fast iteration during development is arguably more important than ever. Paying a compilation cost on every unit test and debug log is painful.

Eager execution avoids expensive graph compilation and instead starts computing results immediately. When you first run Gemma 4 using the new Module your first token appears much more quickly.

Module

Debugging models also gets a lot faster!

Module

The redesigned Module still serves our primary use case, production inference. The built-in compile method traces the model and compiles it as a graph, which lets MAX optimize the computation using knowledge about the whole model. Our graph compiler performs:

Module

compile

Module directly supports both eager and compiled execution, so model authors get the best of both worlds:

Module

model = DeepSeekV3(config)

model.load_state_dict(state_dict)
model(inputs)

compiled = model.compile(*input_types, weights=state_dict)
compiled(inputs)

The redesign was a chance to rethink how we write distributed models. The framework's role is to structure the scaling of a computation from a single GPU to many.

There is no single right way to do that. Many parallelism strategies exist because GPUs differ in compute and memory limits, models differ in layers and dimensions, and inputs differ in shape. Here are several ways to distribute a model:

Lower-level abstractions in MAX can now express data parallelism, pipeline parallelism, or tensor parallelism. A Tensor is still a logical N-D array, but it can now live on more than one device. Operations on distributed tensors behave according to specific sharding rules.

Tensor

A future post will dive deeper into the specifics of our distributed computation support. For now, see below how a row parallel linear layer is implemented:

A Module is now a Tree, a new API in MAX inspired by JAX’s Pytrees. Trees are made of Modules and Tensors which simplifies two things:

Module

Tree

Tree

Module

Tensor

Module.parameters

Module.descendants

Module

Tensor

Module.trace

Module.compile

tree.map

Tensor

These APIs also indirectly help with serialization: tree flatten produces both the flattened leaves and a TreeDef object that stores the tree structure.

flatten

TreeDef

As an example, we use Tree.map to quantize only the expert layers in a mixture-of-experts model.

Tree.map

We also removed abstractions and boilerplate that were no longer needed.

No symbolic Weight class. Weight existed to track and create weights in the graph. Now you use Tensor directly in a Module.

Weight

Weight

Tensor

Module

Default dtype and device. The layer at the top of this post defines neither. A default dtype and device context manager means a model that uses one device or dtype everywhere doesn't need to pass them to every layer. You can still set them per layer when needed.

dtype

device

dtype

device

Built-in compile. Graph construction boilerplate is gone. Every Module now has a compile method, covered in the production inference section above.

compile

Module

compile

The legacy Module implementation now lives in max.nn.legacy and will be removed in Q1 2027, so now is a good time to migrate your existing models. We’re also restructuring max.nn around the new max.tensor and max.functional libraries.

Module

max.nn.legacy

max.nn

max.tensor

max.functional libraries.

The best place to get started is the Model Bringup Workflow tutorial, which walks through building a model end to end with the new APIs.

Looking ahead, 2027 brings quantization, training, and multi-node support. Next in this series is a framework overview showing where MAX fits in the ML framework landscape, followed by posts on eager execution, distributed tensors, serialization, and more.

Modular and Qualcomm: Same code, new silicon

August 18, 2026

Why LLM Inference Needs a New Kind of Router - Part 3

June 5, 2026

Three trends from MLSys 2026

May 29, 2026

Build the future of AI with Modular

Sign up today

Signup to our Cloud Platform today to get started easily.

Browse open models

Browse our model catalog, or deploy your own custom model

Get all our latest news, announcements and updates delivered directly to your inbox. Unsubscribe at anytime.

⚠️ This form requires JavaScript to function. Please enable JavaScript in your browser to continue.

Your data is handled in accordance with Modular's Privacy Policy.

Thanks for signing up to our newsletter! 🚀

Thank you,

Modular Sales Team

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @modular 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/modular-reintroducin…] indexed:0 read:6min 2026-11-19 · —