Introducing the Ettin Reranker Family

The article announces the release of six new Sentence Transformers CrossEncoder reranker models, ranging from 17 million to 1 billion parameters, which are built on Ettin ModernBERT encoders and achieve state-of-the-art performance at their respective sizes. These rerankers, designed for the retrieve-then-rerank pattern in information retrieval, were trained using a distillation recipe and are available for use with just three lines of code via the Sentence Transformers library. The release also includes the full training data and recipe, along with a new agent skill for fine-tuning such models.

Text Ranking • 0.1B • Updated • 680 • 1 Introducing the Ettin Reranker Family Update on GitHub https://github.com/huggingface/blog/blob/main/ettin-reranker.md TL;DR Today I'm releasing six new Sentence Transformers https://sbert.net/ CrossEncoder rerankers, state-of-the-art at their respective sizes, built on top of the Ettin https://huggingface.co/collections/jhu-clsp/encoders-vs-decoders-the-ettin-suite ModernBERT encoders, together with the data and full training recipe that produced them: cross-encoder/ettin-reranker-17m-v1 cross-encoder/ettin-reranker-32m-v1 cross-encoder/ettin-reranker-68m-v1 cross-encoder/ettin-reranker-150m-v1 cross-encoder/ettin-reranker-400m-v1 cross-encoder/ettin-reranker-1b-v1 The models were trained with a distillation recipe : pointwise MSE on mixedbread-ai/mxbai-rerank-large-v2 https://huggingface.co/mixedbread-ai/mxbai-rerank-large-v2 scores over , which is a subset of https://huggingface.co/datasets/cross-encoder/ettin-reranker-v1-data cross-encoder/ettin-reranker-v1-data mixed with a reranked subset of https://huggingface.co/datasets/lightonai/embeddings-pre-training lightonai/embeddings-pre-training . https://huggingface.co/datasets/lightonai/embeddings-fine-tuning lightonai/embeddings-fine-tuning Our six rerankers paired with google/embeddinggemma-300m on MTEB eng, v2 Retrieval. See Results for five more embedder pairings.If you're new to rerankers and want the "why" first, jump to What is a reranker, and why pair one with an embedder? what-is-a-reranker-and-why-pair-one-with-an-embedder . If you just want to plug a model in, jump to Usage usage . If you want to train your own, jump to Training training . I bootstrapped the training recipe below with the new shipped in train-sentence-transformers Agent Skill Sentence Transformers v5.5.0 . Install it with hf skills add train-sentence-transformers --global --claude and ask your AI coding agent Claude Code, Codex, Cursor, Gemini CLI, ... to fine-tune a SentenceTransformer , CrossEncoder , or SparseEncoder model on your data. Table of contents What is a reranker, and why pair one with an embedder? what-is-a-reranker-and-why-pair-one-with-an-embedder Usage usage Architecture Details architecture-details Results results Training training Conclusion conclusion Acknowledgements acknowledgements What is a reranker, and why pair one with an embedder? A reranker a.k.a. pointwise cross-encoder is a neural model that takes a query, document pair and outputs a single relevance score. Unlike an embedding model, which encodes the query and document separately and computes their similarity from the two embedding vectors, a reranker lets the two texts attend to each other through every transformer layer. That joint encoding is more accurate but also more expensive: the model has to be run once per query, document pair rather than once per text. Because cross-encoders are too expensive to run over a full corpus, the common production pattern is retrieve-then-rerank : a fast embedding model retrieves the top-K candidates cheap , then a cross-encoder re-orders just those K with high accuracy. The total cost stays bounded while the final ranking is much closer to what an exhaustive cross-encoder pass would produce. Throughout this blogpost I'll use "reranker" and "cross-encoder" interchangeably. Usage The released models are normal Sentence Transformers CrossEncoder models, so you can use them with just 3 lines of code: python from sentence transformers import CrossEncoder model = CrossEncoder "cross-encoder/ettin-reranker-32m-v1" scores = model.predict "Where was Apple founded?", "Apple Inc. was founded in Cupertino, California in 1976 by Steve Jobs, Steve Wozniak, and Ronald Wayne." , "Where was Apple founded?", "The Fuji apple is an apple cultivar developed in the late 1930s and brought to market in 1962." , print scores 11.393298 2.968891 <- larger means more relevant For a query and a list of candidates, you can also use rank to get back sorted indices and scores: ranked = model.rank query="Which planet is known as the Red Planet?", documents= "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.", , top k=4, return documents=True, for r in ranked: print f" {r 'score' :.2f} : {r 'text' }" 10.82 : Mars, known for its reddish appearance, is often referred to as the Red Planet. 9.86 : Saturn, famous for its rings, is sometimes mistaken for the Red Planet. 8.55 : Jupiter, the largest planet in our solar system, has a prominent red spot. 6.21 : Venus is often called Earth's twin because of its similar size and proximity. You can swap cross-encoder/ettin-reranker-32m-v1 https://huggingface.co/cross-encoder/ettin-reranker-32m-v1 for any other size to trade quality for speed. All six accept up to 8K tokens of context useful for long-document reranking thanks to ModernBERT's long-context pre-training. It is recommended to install kernels https://github.com/huggingface/kernels and set model kwargs={"dtype": "bfloat16", "attn implementation": "flash attention 2"} for the highest throughput. See the Speed speed section below for more details, but in general you can expect a 1.7x-8.3x speedup over default loading depending on model size and sequence length. python from sentence transformers import CrossEncoder model = CrossEncoder "cross-encoder/ettin-reranker-32m-v1", model kwargs={"dtype": "bfloat16", "attn implementation": "flash attention 2"}, End-to-end retrieve-then-rerank pipeline A complete example with a fast embedder for retrieval and the reranker for the final ordering: python from sentence transformers import SentenceTransformer, CrossEncoder Fast retrieval with a static embedder sub-millisecond on CPU per query embedder = SentenceTransformer "sentence-transformers/static-retrieval-mrl-en-v1" reranker = CrossEncoder "cross-encoder/ettin-reranker-68m-v1" corpus = "Apple Inc. was founded in Cupertino, California in 1976 by Steve Jobs, Steve Wozniak, and Ronald Wayne.", "The Fuji apple is an apple cultivar developed in the late 1930s.", "Steve Jobs introduced the iPhone in 2007 at Macworld.", "Macintosh computers were sold by Apple from 1984 onward.", ... thousands or millions more in production query = "Where was Apple founded?" Step 1: encode + retrieve top-100 query emb = embedder.encode query query, convert to tensor=True corpus emb = embedder.encode document corpus, convert to tensor=True scores = embedder.similarity query emb, corpus emb 0 top k idx = scores.topk min 100, len corpus .indices.tolist Step 2: rerank top k docs = corpus i for i in top k idx ranked = reranker.rank query, top k docs, top k=5, return documents=True for r in ranked: print f" {r 'score' :.2f} : {r 'text' }" 11.63 : Apple Inc. was founded in Cupertino, California in 1976 by Steve Jobs, Steve Wozniak, and Ronald Wayne. 4.71 : Steve Jobs introduced the iPhone in 2007 at Macworld. 1.96 : The Fuji apple is an apple cultivar developed in the late 1930s. 1.49 : Macintosh computers were sold by Apple from 1984 onward. This is the same shape used by most modern search systems. The retriever decides what enters the funnel, the reranker decides what wins. Architecture Details All six rerankers share the same architecture and differ only in their backbone size. The backbone is one of the six Ettin encoders https://huggingface.co/blog/ettin from Johns Hopkins University's Ettin suite. These are ModernBERT-style models with unpadded attention, RoPE positional encodings, GeGLU, and 2T tokens of open-license pre-training, supporting up to 8192 tokens of context. On top of each encoder, the reranker uses a 4-module classification head that mirrors ModernBertForSequenceClassification but is built from Sentence Transformers' modular components. The underlying Transformer is a plain AutoModel rather than AutoModelForSequenceClassification , which lets us use sequence unpadding for variable-length inputs for Flash Attention 2. At medium-document sequence lengths this is a 1.7x-8.3x speedup over fp32+SDPA depending on model size see Speed speed for the full benchmark : 1. Transformer FA2 2. Pooling cls 3. Dense H, H, bias=False, GELU 4. LayerNorm H 5. Dense H, 1, scores In my ablations, CLS pooling outperformed mean pooling. That was a little surprising. ModernBERT uses global attention only every third layer and the other two-thirds use local-window attention that cannot reach CLS from distant positions. Empirically, those few global layers carry enough signal to make CLS the better pooling choice. | Model | Backbone | Hidden size | Layers | Params head incl. | |---|---|---|---|---| cross-encoder/ettin-reranker-17m-v1 | jhu-clsp/ettin-encoder-17m cross-encoder/ettin-reranker-32m-v1 jhu-clsp/ettin-encoder-32m cross-encoder/ettin-reranker-68m-v1 jhu-clsp/ettin-encoder-68m cross-encoder/ettin-reranker-150m-v1 jhu-clsp/ettin-encoder-150m cross-encoder/ettin-reranker-400m-v1 jhu-clsp/ettin-encoder-400m cross-encoder/ettin-reranker-1b-v1 jhu-clsp/ettin-encoder-1b All six models are released under the Apache 2.0 license, matching the Ettin encoders. Results MTEB eng, v2 Retrieval I ran each released model through the full MTEB eng, v2 Retrieval benchmark https://github.com/embeddings-benchmark/mteb 10 tasks, top-100 reranked using MTEB's two-stage reranking flow https://embeddings-benchmark.github.io/mteb/get started/advanced usage/two stage reranking/ , pairing each reranker with six embedding models that span the speed/quality spectrum: | Embedding Model | Active params | Retriever-only NDCG@10 | |---|---|---| sentence-transformers/static-retrieval-mrl-en-v1 | sentence-transformers/all-MiniLM-L6-v2 BAAI/bge-small-en-v1.5 nomic-ai/nomic-embed-text-v1.5 google/embeddinggemma-300m jinaai/jina-embeddings-v5-text-small-retrieval The dashed retriever-only line in each chart below is the headline number to beat. Anything below it means the reranker actively hurts the pipeline on average: Full table of results click to expand Mean NDCG@10 over the 6 embedder pairings, sorted descending. Our six models are in bold , and the teacher mixedbread-ai/mxbai-rerank-large-v2 https://huggingface.co/mixedbread-ai/mxbai-rerank-large-v2 is underlined. † Capped to max seq length=8192 the 4B Qwen3-based rerankers don't fit on a single H100 80GB at native context . Native-context evaluation is likely higher. Full table of NanoBEIR results click to expand NanoBEIR https://huggingface.co/collections/sentence-transformers/nanobeir-with-bm25-rankings is a fast 13-dataset subset of BEIR https://github.com/beir-cellar/beir that uses 50 queries per dataset against up to 5000 documents each. NanoBEIR is what metric for best model was set to during training see Evaluation evaluation , and what I used to guide the experimentation. | Reranker | Params | NanoBEIR mean NDCG@10 | |---|---|---| mixedbread-ai/mxbai-rerank-large-v2 | cross-encoder/ettin-reranker-1b-v1 1.00B 0.7237 jinaai/jina-reranker-m0 cross-encoder/ettin-reranker-400m-v1 401M 0.7193 mixedbread-ai/mxbai-rerank-base-v2 cross-encoder/ettin-reranker-150m-v1 151M 0.7086 Alibaba-NLP/gte-reranker-modernbert-base BAAI/bge-reranker-v2-m3 cross-encoder/ettin-reranker-68m-v1 68.6M 0.6915 ibm-granite/granite-embedding-reranker-english-r2 cross-encoder/ettin-reranker-32m-v1 32.8M 0.6825 cross-encoder/ettin-reranker-17m-v1 17.6M 0.6746 mixedbread-ai/mxbai-rerank-large-v1 BAAI/bge-reranker-large cross-encoder/ms-marco-MiniLM-L12-v2 cross-encoder/ms-marco-MiniLM-L6-v2 cross-encoder/ms-marco-MiniLM-L4-v2 mixedbread-ai/mxbai-rerank-base-v1 mixedbread-ai/mxbai-rerank-xsmall-v1 BAAI/bge-reranker-base The smallest model I'm releasing, our 17M, beats the 33M ms-marco-MiniLM-L12-v2 by +0.051 NDCG@10 0.5576 vs 0.5066 on MTEB and +0.038 0.6746 vs 0.6369 on NanoBEIR at roughly half the parameter count. The 32M beats the 568M BAAI/bge-reranker-v2-m3 by +0.025 0.5779 vs 0.5526 on MTEB, a 17x parameter gap. If you've been using one of the legacy MiniLM rerankers as the default in your retrieve-then-rerank stack, swapping in our 17M or 32M is a low-risk drop-in replacement, with a noticeable quality bump on both benchmarks. Moving up the table, our 150M is the strongest reranker I tested in the under-600M range on MTEB, edging out the recent Qwen/Qwen3-Reranker-0.6B 596M by +0.005 0.5994 vs 0.5940 and beating every BAAI bge-reranker variant by 0.03 to 0.05. The 68M is also worth a mention: at 0.5915 it lands almost exactly on Qwen3-Reranker-0.6B 0.5940 while using a ninth of the parameters. At the top of the released range, our 1B model closely tracks its teacher. It comes within 0.0001 of the 1.54B mxbai-rerank-large-v2 on MTEB 0.6114 vs 0.6115 and within 0.008 on NanoBEIR, despite distilling from a model 54% larger than itself. The distillation effectively closes the gap to the teacher, which is what I was hoping to see going into this release. The overall strongest reranker in the comparison is Qwen/Qwen3-Reranker-4B at 0.6367 MTEB, +0.025 above our 1B model. Closing that gap from the current recipe would likely require distilling from a stronger teacher our teacher itself sits below Qwen3-Reranker-4B . For most retrieve-then-rerank workloads, our 1B at a quarter of the parameters see Speed speed is a much more practical pick. Speed Quality numbers are only half of what matters for a reranker. The other half is whether its latency fits inside the budget you have between retrieval and showing results to the user. Let me walk through what I measured. I benchmarked all six released models against thirteen public rerankers strong baselines up to about 1B parameters on a single NVIDIA H100 80GB. The queries and documents come from sentence-transformers/natural-questions https://huggingface.co/datasets/sentence-transformers/natural-questions at its natural document-length distribution: most NQ answers are short, some are long. Documents are truncated at max length=512 to avoid giving the older models an unfair advantage. Each model uses its best supported attention implementation: Flash Attention 2 wherever the architecture supports it BERT, XLM-RoBERTa, ModernBERT, Qwen2 , SDPA where it doesn't, and eager for DeBERTa-v2 which currently has neither FA2 nor SDPA support in transformers .For every model an auto-batch search starts at batch size 8 and doubles until the GPU runs out of memory. At each batch size I run three timed passes and keep the median throughput, so a single unlucky run doesn't drag the number around. The reported throughput is at whichever batch size won. Table 1. Throughput in pairs per second, all in bfloat16 . Our six rerankers are in bold . | Model | Params | Attn | pairs / second | |---|---|---|---| cross-encoder/ettin-reranker-17m-v1 | 17M | FA2 | 7517 | cross-encoder/ettin-reranker-32m-v1 | 32M | FA2 | 6602 | cross-encoder/ettin-reranker-68m-v1 | 68M | FA2 | 4913 | cross-encoder/ms-marco-MiniLM-L4-v2 | cross-encoder/ms-marco-MiniLM-L6-v2 cross-encoder/ms-marco-MiniLM-L12-v2 cross-encoder/ettin-reranker-150m-v1 150M 3237 BAAI/bge-reranker-base mixedbread-ai/mxbai-rerank-xsmall-v1 mixedbread-ai/mxbai-rerank-base-v1 cross-encoder/ettin-reranker-400m-v1 400M 1738 BAAI/bge-reranker-large BAAI/bge-reranker-v2-m3 Alibaba-NLP/gte-reranker-modernbert-base ibm-granite/granite-embedding-reranker-english-r2 cross-encoder/ettin-reranker-1b-v1 1B 928 mixedbread-ai/mxbai-rerank-large-v1 mixedbread-ai/mxbai-rerank-base-v2 mixedbread-ai/mxbai-rerank-large-v2 1.5B 387 Our 17M is the fastest reranker in the whole comparison, at 7517 pairs per second. That's almost twice the throughput of ms-marco-MiniLM-L6-v2 3817 and faster even than the smaller ms-marco-MiniLM-L4-v2 4029 . And as you saw in the MTEB table earlier, our 17M is also more accurate than every MiniLM variant. If you're currently running a MiniLM cross-encoder, swapping to our 17M is a one-line change that improves both your latency and search quality. Our 150M is an even more interesting comparison, because there are two direct architectural peers at exactly 150M parameters: Alibaba-NLP/gte-reranker-modernbert-base https://huggingface.co/Alibaba-NLP/gte-reranker-modernbert-base and . Both are built on the same ModernBERT-base backbone. Our 150M runs at 3237 pairs per second, while the two peers come in at 1418 and 1404 respectively, for a 2.3x speed gap. https://huggingface.co/ibm-granite/granite-embedding-reranker-english-r2 ibm-granite/granite-embedding-reranker-english-r2 All three 150M models use Flash Attention 2, but the two peers load through AutoModelForSequenceClassification , which keeps the inputs padded. So attention itself runs the FA2 kernel, but the rest of the model is still doing dense compute on padding tokens that don't contribute anything. Our modular Transformer module see Architecture Details architecture-details above propagates unpadded inputs all the way through the model, so every layer only spends compute on real tokens. That's the difference between getting some of FA2's benefit and getting all of it. At the bottom of the table, our 1B model hits 928 pairs per second, which is 2.4x faster than the 1.54B teacher mxbai-rerank-large-v2 387 pairs per second while matching its MTEB score within 0.0001. The teacher is Qwen2-based with a prompt-template overhead per pair, so the distilled student inherits the teacher's calibration and judgement but skips all the runtime baggage. This is honestly the most satisfying single number in the whole release for me. One unfortunate note: the DeBERTa-v2-based mxbai-rerank-{xsmall,base,large}-v1 series ends up much slower than the rest of the table because DeBERTa-v2 currently supports neither Flash Attention 2 nor SDPA in transformers . The 70M mxbai-rerank-xsmall-v1 runs at 2636 pairs per second, about half the throughput of our 68M at almost the same parameter count. The models themselves are perfectly fine, they just don't get to use modern attention kernels. Same benchmark on a consumer GPU RTX 3090, 24 GB If you're self-hosting on a consumer card rather than a datacenter GPU, here's the same throughput sweep on an RTX 3090. Same benchmark setup as Table 1: bfloat16 , best-supported attention per model, three-trial median throughput at the largest batch that fits. | Model | Params | Best attn | pairs / second | |---|---|---|---| cross-encoder/ettin-reranker-17m-v1 | 17M | FA2 | 9008 | cross-encoder/ms-marco-MiniLM-L4-v2 | cross-encoder/ettin-reranker-32m-v1 32M 4497 cross-encoder/ms-marco-MiniLM-L6-v2 cross-encoder/ms-marco-MiniLM-L12-v2 cross-encoder/ettin-reranker-68m-v1 68M 1916 mixedbread-ai/mxbai-rerank-xsmall-v1 BAAI/bge-reranker-base cross-encoder/ettin-reranker-150m-v1 150M 982 mixedbread-ai/mxbai-rerank-base-v1 ibm-granite/granite-embedding-reranker-english-r2 Alibaba-NLP/gte-reranker-modernbert-base BAAI/bge-reranker-large BAAI/bge-reranker-v2-m3 cross-encoder/ettin-reranker-400m-v1 400M 429 mixedbread-ai/mxbai-rerank-large-v1 mixedbread-ai/mxbai-rerank-base-v2 cross-encoder/ettin-reranker-1b-v1 1B 189 mixedbread-ai/mxbai-rerank-large-v2 1.5B 69 Our 17M is still the fastest model in the table at 9008 pairs per second, actually higher than its H100 number, which suggests that at tiny sizes raw compute isn't the bottleneck and the H100's extra muscle doesn't translate. The middle of the table reshuffles a bit, with the MiniLM rerankers overtaking our 32M and 68M, and the 1B slipping behind mxbai-rerank-base-v2 189 vs 221 pairs per second . Our 150M model still holds a solid lead over the two 150M ModernBERT-based peers, and the teacher-replacement story still holds, with our 1B at 2.7x the throughput of the 1.5B mxbai-rerank-large-v2 189 vs 69 pairs per second . Same benchmark on CPU Intel Core i7-13700K | Model | Params | Best attn | pairs / second | |---|---|---|---| cross-encoder/ettin-reranker-17m-v1 | 17M | SDPA | 267.4 | cross-encoder/ms-marco-MiniLM-L4-v2 | cross-encoder/ms-marco-MiniLM-L6-v2 cross-encoder/ettin-reranker-32m-v1 32M 92.5 cross-encoder/ms-marco-MiniLM-L12-v2 mixedbread-ai/mxbai-rerank-xsmall-v1 cross-encoder/ettin-reranker-68m-v1 68M 31.2 BAAI/bge-reranker-base Alibaba-NLP/gte-reranker-modernbert-base ibm-granite/granite-embedding-reranker-english-r2 cross-encoder/ettin-reranker-150m-v1 150M 14.0 mixedbread-ai/mxbai-rerank-base-v1 BAAI/bge-reranker-large BAAI/bge-reranker-v2-m3 cross-encoder/ettin-reranker-400m-v1 400M 5.2 mixedbread-ai/mxbai-rerank-large-v1 mixedbread-ai/mxbai-rerank-base-v2 cross-encoder/ettin-reranker-1b-v1 1B 2.1 On CPU, we can't take advantage of bf16, Flash Attention 2, or unpadding, so the latency story is a bit simpler: the higher the parameter count, the slower the model. The 17M model is considerably faster than ms-marco-MiniLM-L6-v2 267.4 vs 143.9 pairs per second and even faster than the smaller ms-marco-MiniLM-L4-v2 206.2 . As expected, our 150M model lands alongside the two 150M peers 14.0 vs 14.5 and 14.7 pairs per second now that unpadding no longer applies. If you're CPU-bound, our 17M and 32M are the practical picks. To explain where the speed comes from, the next table sweeps fp32+SDPA , bf16+SDPA , and bf16+FA2 for our six models using the same bench config. The FA2 column is split in two: one with the inputs still padded what a wrapped model would see and one with unpadded inputs what our modular Transformer actually does . The rightmost column is what our models use by default when FA2 is enabled. Table 2. Precision and attention ablation for the six released sizes at max length=512 on natural NQ documents. Each cell shows pairs / second with the multiplier relative to fp32+SDPA in parentheses, and peak GPU memory on the second line. The rightmost column in bold is the configuration our models use by default when FA2 is enabled. | Model | Params | fp32+SDPA | bf16+SDPA | bf16+FA2 w. padding | bf16+FA2 w.o. padding | |---|---|---|---|---|---| cross-encoder/ettin-reranker-17m-v1 | 0.8 GB 2.2 GB 1.9 GB 7517 1.71x 1.4 GB cross-encoder/ettin-reranker-32m-v1 1.2 GB 1.6 GB 2.9 GB 6602 2.00x 1.1 GB cross-encoder/ettin-reranker-68m-v1 1.0 GB 2.2 GB 2.0 GB 4913 3.60x 1.5 GB cross-encoder/ettin-reranker-150m-v1 1.6 GB 1.8 GB 3.1 GB 3237 4.83x 1.4 GB cross-encoder/ettin-reranker-400m-v1 2.5 GB 1.8 GB 2.7 GB 1738 6.53x 2.2 GB cross-encoder/ettin-reranker-1b-v1 4.6 GB 2.8 GB 3.6 GB 928 8.26x 4.5 GB The total speedup from bf16+FA2 w.o. padding over the fp32+SDPA baseline grows sharply with model size, from 1.71x on the 17M to 8.26x on the 1B. Most of that growth comes from bf16 alone: the fp32+SDPA to bf16+SDPA step gives the 17M only a 1.03x speedup but gives the 1B a full 5.60x speedup, also due to the lowered memory cost allowing for bigger batch sizes. In short, bfloat16 is the biggest single contributor to the overall speedup. Unexpectedly, turning on FA2 while the inputs are still padded is actually slower than bf16+SDPA at every size in the release. The FA2 kernel prefers an unpadded format, and when you feed it padded inputs you pay the bookkeeping overhead of converting between formats while still spending compute on the padding tokens themselves. So the bf16+FA2 w. padding column is roughly what you'd measure if you swapped sdpa for flash attention 2 in model kwargs without changing anything else about the model loader. This is the situation that gte-reranker-modernbert-base and granite-embedding-reranker-english-r2 from Table 1 are in. Lastly, going from bf16+FA2 w. padding to bf16+FA2 w.o. padding is worth between 1.78x 1B and 2.45x 68M of additional throughput, and it also cuts peak memory considerably, allowing for higher batch sizes. So my recommendation is simple: enable bf16 and FA2 together. The six Ettin rerankers will use unpadded inputs by default, since that's what the modular Transformer module from the Architecture Details architecture-details section is set up for. The full snippet is the same as in the Usage usage section above: python from sentence transformers import CrossEncoder model = CrossEncoder "cross-encoder/ettin-reranker-150m-v1", model kwargs={ "dtype": "bfloat16", "attn implementation": "flash attention 2", See tip below }, Use pip install kernels to install FA2. It ships pre-built kernels for a wide range of GPU architectures, CUDA versions, and operating systems. One caveat for other CrossEncoders: the full speedup is only available for models built with a modular Transformer like the Ettin rerankers. Applying the same two flags to a CrossEncoder that loads through AutoModelForSequenceClassification lands you in the slower bf16+FA2 w. padding column of Table 2 instead. Training The training script below started as the output of the new train-sentence-transformers Agent Skill https://github.com/huggingface/sentence-transformers/tree/main/skills , shipped in Sentence Transformers v5.5.0 https://github.com/huggingface/sentence-transformers/releases/tag/v5.5.0 . If you use an AI coding agent Claude Code, Codex, Cursor, Gemini CLI, ... , you can install the skill and ask it to fine-tune a SentenceTransformer , CrossEncoder , or SparseEncoder model on your data. The skill carries version-aware guidance for base model selection, loss and evaluator choice, hard-negative mining, distillation, LoRA, Matryoshka, multilingual training, and static embeddings, plus template scripts for each model type. hf skills add train-sentence-transformers --claude symlinks into .claude/skills/ hf skills add train-sentence-transformers --global under ~/.agents/skills/ A prompt like "Fine-tune a cross-encoder reranker on query, document pairs from my dataset, mine hard negatives, and push to my Hub repo" will produce a runnable script you can then iterate on. That's how I started working on the recipe below. All six rerankers were trained with the same single-stage recipe. Only the learning rate and the per-device batch size vary per model size. The full training script is ~150 lines and uses one published dataset. The recipe converged after a single sweep across model sizes. Each size's learning rate was tuned by a small grid search on a ~15% subset of the final training data, and the resulting LRs transferred cleanly to the full-data runs without re-tuning. No per-size tuning beyond LR was needed. Distillation recipe Most published reranker recipes train on human-labeled relevance triples a query, one positive document, and optionally hard negatives with a contrastive, pointwise, pairwise, or listwise loss like MultipleNegativesRankingLoss https://sbert.net/docs/package reference/cross encoder/losses.html multiplenegativesrankingloss , , https://sbert.net/docs/package reference/cross encoder/losses.html binarycrossentropyloss BinaryCrossEntropyLoss , or https://sbert.net/docs/package reference/cross encoder/losses.html ranknetloss RankNetLoss , respectively. See my earlier https://sbert.net/docs/package reference/cross encoder/losses.html lambdaloss LambdaLoss Training and Finetuning Reranker Models with Sentence Transformers https://huggingface.co/blog/train-reranker blogpost, for example. But this approach has a few practical and theoretical drawbacks. First, positives need to be human-labeled, which is expensive and slow to scale across many domains. Second, the model only ever sees a label for the small subset of query, document pairs that someone went through. Especially after hard negative mining, you end up with a lot of false negatives, e.g. as shown in Hard Negatives, Hard Lessons https://arxiv.org/abs/2505.16967 . Third, the binary nature of this labeling doesn't match reality, where some documents are simply more relevant than others. I took a different route here: pointwise MSE distillation from an existing strong teacher reranker. The setup is simple enough to describe in three lines: Teacher : 1.54B parameters . mixedbread-ai/mxbai-rerank-large-v2 Loss :on the raw teacher logits range ~ −12, 22 , i.e. without rescaling. MSELoss Training data : ~143M query, document, teacher score triples. Dataset I've released the training data as a single Hugging Face dataset, cross-encoder/ettin-reranker-v1-data https://huggingface.co/datasets/cross-encoder/ettin-reranker-v1-data , assembled from two sources. Each source is kept as its own split so the provenance is transparent: - LightOn pre-training data , non-curated : 32 splits covering broad-domain text similarity signal MTP, FW-EDU, Reddit, PAQ, S2ORC, Amazon, Wikipedia, MS MARCO, etc. . I limit the number of samples for some of the splits, resulting in ~110M lightonai/embeddings-pre-training query, document, similarity triples in total. - Rescored retrieval data from : 7 splits lightonai/embeddings-fine-tuning msmarco , hotpotqa , trivia , nq , squadv2 , fiqa , fever . The source dataset has up to 2048 candidate documents per query initially scored with , which I rescored with Alibaba-NLP/gte-modernbert-base and uploaded as mixedbread-ai/mxbai-rerank-large-v2 . That dataset subsamples each query's 2048 candidates down to 256 using the cross-encoder/lightonai-embeddings-fine-tuning-reranked-v1 Jang et al. https://arxiv.org/abs/2604.04734 quantile-anchor recipe all positives + top-16 hard + ~239 quantile-anchor stratified . For training, I pick 64 of those 256 per query: 32 from the score-sorted head the positive plus the hardest negatives and 32 medium-difficulty negatives sampled from a band further down the teacher's ranking. See the dataset card https://huggingface.co/datasets/cross-encoder/ettin-reranker-v1-data for the exact rank positions. Total: ~143M query, document, score triples, plus a held-out 5K-row eval split the tail of quora that drives the in-training eval loss. Training Arguments Most hyperparameters are constant across model sizes: CrossEncoderTrainingArguments num train epochs=1, I chose more data over more epochs per device train batch size=..., global batch size // world size see table below gradient accumulation steps=1, learning rate=..., per-size, see table warmup ratio=0.03, ~3% linear warmup, then linear decay default bf16=True, FA2 + bf16 throughout eval strategy="steps", eval steps=0.05, NanoBEIR every 5% of training save strategy="steps", save steps=0.05, save total limit=5, load best model at end=True, metric for best model="eval NanoBEIR R100 mean ndcg@10", seed=12, Only the learning rate and global batch size very per model size. | Size | Learning rate | Global batch size | |---|---|---| | 17m | 2.4e-4 | 1024 | | 32m | 1.2e-4 | 512 | | 68m | 3e-5 | 256 | | 150m | 1.5e-5 | 192 | | 400m | 7e-6 | 256 | | 1b | 3e-6 | 512 | global batch size is per device batch size x world size x gradient accumulation steps . On a single 8-GPU node, the 1024 global batch for 17m means per device=128 . On 8 nodes, it means per device=8 . The training script computes per device batch size from global batch size // world size so the same script works at any node count. The global batch size could be made more consistent, but I found that the above values worked well and didn't want to retune them just for the sake of consistency. Evaluation I monitored NanoBEIR mean NDCG@10 during training eval every 5% of steps and used it as the metric for best model for load best model at end . NanoBEIR is fast, so I could afford it 20 times per training run. After training, I evaluated both the best checkpoint according to NanoBEIR and the last checkpoint on the full MTEB eng, v2 Retrieval benchmark. The final release checkpoint was the one that did best on MTEB. The NanoBEIR-preferred checkpoint won for all sizes except 68m, where the last checkpoint was slightly stronger. Overall Training Script The complete script what every released model was trained with is a single file. Only ENCODER SIZE changes per run, and everything else is automatic: python from future import annotations import logging import os from pathlib import Path import torch import torch.nn as nn from datasets import concatenate datasets, get dataset config names, load dataset from sentence transformers import CrossEncoder from sentence transformers.base.modules import Dense from sentence transformers.cross encoder import CrossEncoderModelCardData, CrossEncoderTrainer, CrossEncoderTrainingArguments, from sentence transformers.cross encoder.evaluation import CrossEncoderNanoBEIREvaluator from sentence transformers.cross encoder.losses import MSELoss from sentence transformers.sentence transformer.modules import LayerNorm, Pooling, Transformer logging.basicConfig level=logging.INFO, format="% asctime s % message s", datefmt="%H:%M:%S" logging.getLogger "httpx" .setLevel logging.WARNING Per-size config. I swept the learning rates with these global effective batch sizes, also by incorporating accum steps CONFIGS: dict str, dict = { "17m": {"base model name": "jhu-clsp/ettin-encoder-17m", "learning rate": 2.4e-4, "global batch size": 1024}, "32m": {"base model name": "jhu-clsp/ettin-encoder-32m", "learning rate": 1.2e-4, "global batch size": 512}, "68m": {"base model name": "jhu-clsp/ettin-encoder-68m", "learning rate": 3e-5, "global batch size": 256}, "150m": {"base model name": "jhu-clsp/ettin-encoder-150m", "learning rate": 1.5e-5, "global batch size": 192}, "400m": {"base model name": "jhu-clsp/ettin-encoder-400m", "learning rate": 7e-6, "global batch size": 256}, "1b": {"base model name": "jhu-clsp/ettin-encoder-1b", "learning rate": 3e-6, "global batch size": 512}, } ENCODER SIZE = "17m" def main - None: config = CONFIGS ENCODER SIZE encoder id = config "base model name" learning rate = config "learning rate" global batch size = config "global batch size" world size = int os.environ.get "WORLD SIZE", 1 per device batch size = global batch size // world size dataloader workers = 0 if world size 8 else 4 run name = f"ettin-reranker-{ENCODER SIZE}-lr{learning rate:.0e}" 1. Load a model to finetune with model card data The model mirrors ModernBertForSequenceClassification, but with a 'headless' Transformer that just loads AutoModel. This allows for unpadding with FA2, which isn't possible with AutoModelForSequenceClassification. This speeds up training considerably, while heavily reducing memory usage. torch.manual seed 12 transformer = Transformer encoder id, model kwargs={"attn implementation": "flash attention 2"} transformer.model.config.num labels = 1 embedding dimension = transformer.get embedding dimension pooling = Pooling embedding dimension=embedding dimension, pooling mode="cls" dense inner = Dense in features=embedding dimension, out features=embedding dimension, bias=False, activation function=nn.GELU , module input name="sentence embedding", module output name="sentence embedding", norm = LayerNorm dimension=embedding dimension dense score = Dense in features=embedding dimension, out features=1, bias=True, activation function=nn.Identity , module input name="sentence embedding", module output name="scores", model = CrossEncoder modules= transformer, pooling, dense inner, norm, dense score , num labels=1, activation fn=nn.Identity , model card data=CrossEncoderModelCardData model name=f"Ettin Reranker {ENCODER SIZE} distilled from mxbai-rerank-large-v2", language="en", license="apache-2.0", , actual attn = getattr model 0 .model.config, " attn implementation", None if not actual attn and "flash" in actual attn.lower : logging.warning f"FA2 may not be active attn impl={actual attn r} ; training will be slower." 2. Load the dataset. Each config is one source subset 32 lighton + 7 rerank retrieval domains . The held-out eval rows live as the 'validation' split of the 'quora' config. dataset repo = "cross-encoder/ettin-reranker-v1-data" train pieces = eval dataset = None for config name in get dataset config names dataset repo : dataset = load dataset dataset repo, config name train pieces.append dataset "train" if "validation" in dataset: eval dataset = dataset "validation" train dataset = concatenate datasets train pieces print train dataset 3. Define a loss function loss = MSELoss model 4. Specify training arguments args = CrossEncoderTrainingArguments output dir=f"models/{run name}", num train epochs=1, per device train batch size=per device batch size, per device eval batch size=per device batch size, gradient accumulation steps=1, learning rate=learning rate, warmup ratio=0.03, bf16=True, eval strategy="steps", eval steps=0.05, save strategy="steps", save steps=0.05, save total limit=5, logging steps=0.025, logging first step=True, load best model at end=True, metric for best model="eval NanoBEIR R100 mean ndcg@10", dataloader num workers=dataloader workers, run name=run name, seed=12, 5. Create an evaluator evaluator = CrossEncoderNanoBEIREvaluator dataset names= "msmarco", "nfcorpus", "nq", "fiqa2018", "touche2020", "scifact", "hotpotqa", "arguana", "fever", "dbpedia", "climatefever", "scidocs", "quoraretrieval" , batch size=per device batch size, always rerank positives=False, show progress bar=False, 6. Create a trainer trainer = CrossEncoderTrainer model=model, args=args, train dataset=train dataset, eval dataset=eval dataset, loss=loss, evaluator=evaluator, 7. Evaluate before training if trainer.is world process zero : with torch.autocast device type="cuda", dtype=torch.bfloat16 : evaluator model 8. Train trainer.train 9. Evaluate the final model if trainer.is world process zero : with torch.autocast device type="cuda", dtype=torch.bfloat16 : evaluator model 10. Save the final model final dir = f"models/{run name}/final" model.save pretrained final dir if name == " main ": main For multi-node training anything past 17m/32m , launch the same script with torchrun : Single-node 17m, 32m : defaults work python train.py Multi-node 4n setup for 150m, preserves global batch size=192: torchrun --nproc per node=8 --nnodes=4 ... train.py Conclusion The ettin-reranker-v1 family, trained with a single simple recipe, is state-of-the-art at every released size up to 1B parameters. Pointwise MSE distillation from a strong teacher onto a broad-domain and retrieval-specific mix scales cleanly from 17M to 1B parameters, with only the learning rate and per-device batch size changing between sizes. Every ettin-reranker-v1 model beats the ms-marco-MiniLM-L -v2 family by a comfortable margin on MTEB and NanoBEIR. cross-encoder/ettin-reranker-150m-v1 is the strongest mid-tier reranker I tested in the under-600M range, cross-encoder/ettin-reranker-400m-v1 lands within 0.0024 of the 1.54B teacher's MTEB score, and cross-encoder/ettin-reranker-1b-v1 matches that teacher within 0.0001. Everything in one place: Models : Dataset :with ~143M cross-encoder/ettin-reranker-v1-data query, document, label triples, kept as 39 named splits so the provenance of every row is visible. Training script : the ~150 lines in Overall Training Script overall-training-script above, which is the same script used for all six models. If you build something on top of these, please let me know I'd genuinely love to see what people do with them, and if you manage to train better rerankers using the released data, even better. The recipe is intentionally simple, partly so that there's plenty of headroom for someone else to improve it. Train a stronger teacher and the same script can keep producing better students. Acknowledgements I'd like to thank the Ettin team Orion Weller, Kathryn Ricci, Marc Marone, Antoine Chaffin, Dawn Lawrie, and Benjamin Van Durme for building the base encoders https://huggingface.co/blog/ettin that these rerankers are built on, the LightOn team Antoine Chaffin, Raphael Sourty, Paulo Moura, and Amélie Chatelain for their work on the training data collection https://huggingface.co/blog/lightonai/denseon-lateon , and the Mixedbread AI team Xianming Li, Aamir Shakir, Rui Huang, Tsz-fung Andrew Lee, Julius Lipp, Benjamin Clavié, and Jing Li for their work on the teacher model https://arxiv.org/abs/2506.03487 . Citation If you use the ettin-reranker-v1 family or any of the released artifacts, please cite this blogpost: @misc{aarsen2026ettin-reranker, title = "Introducing the Ettin Reranker Family", author = "Aarsen, Tom", year = "2026", publisher = "Hugging Face", url = "https://huggingface.co/blog/ettin-reranker", }