Fern is Mirendil’s first model. It is pretrained from scratch and supports a context window of up to 10M tokens. Its first version, Fern 0.1, was finished with a short round of post-training. New versions ship frequently.
The Mirendil grove #
Fern
0.1 · Latest Our most efficient model, built for focused, high-volume agentic tasks.
- Base model
- Mirendil-Base
- Size
- 0.1–0.5T params
- Context
- Up to 10M tokens
- Input
- Text and code
- Built for
- High-volume agentic tasks
- Access
- Coming soon
More capable, for harder and longer-running work.
- Base model
- Mirendil-Base-Large
- Size
- 0.5–2T params
- Context
- —
- Input
- —
- Built for
- —
- Access
- —
Our most capable model, for the most demanding work.
- Base model
- Mirendil-Base-XL
- Size
- 2–10T params
- Context
- —
- Input
- —
- Built for
- —
- Access
- —
Context is intelligence #
A language model’s capabilities grow with compute, and there are two ways to spend more of it. One is the model’s parameter count. The other is its context length. A model with more parameters holds more static knowledge and can carry out more sophisticated reasoning steps. A model with a larger context window can use more task-specific information: draw on more of its own past experience, keep track of more specifications and instructions, integrate evidence from more documents and more code, and look at more data directly.
Fern’s architecture, training and serving can support up to 10M tokens, about 23 MB
of code, at once. That’s about half a million lines of code, a codebase on the
scale of the Megatron-LM library, dozens to hundreds of complete agent traces, or days of
an agent’s work, all held in context *at the same time*.
Below, we first show that more context helps even with next-token prediction, the foundation of language model training and of every capability built on it. We then run a basic test of effective access to the long context with a standard benchmark (RULER). More comprehensive long-context results are coming soon.
Early post-training results #
Fern is built for the focused, high-volume agentic work of anyone building AI: auditing training data, debugging a training run from its logs, and reading agent traces end to end. Fern 0.1 is Mirendil-Base after a short round of post-training. Here’s how it performs on agentic benchmarks at this point.
What Fern is for #
-
Works on larger reposA 10M-token context window, so it can work across larger codebases.
-
Agentic codingRuns commands, reads the output and keeps going until the tests pass.
-
Research codeSpecialized for the code behind AI models: training loops, data pipelines and evaluations.
-
Debugging from logsWorks through long logs and stack traces to find where a run went wrong.
-
Analyzing full tracesReads an entire agent or training trace end to end, so it can explain what happened and why.
-
Auditing training dataScans a whole pretraining shard for duplicates, benchmark contamination and junk.
1 Bits per byte is a model’s cross-entropy loss per byte of text, so models with different tokenizers can be compared fairly. Lower is better.