Show HN: Fastembed-rs – Rust library for generating vector embeddings, reranking

Fastembed-rs, a Rust library for generating vector embeddings and reranking, has been released. It supports synchronous usage without Tokio dependency, uses ONNX inference via pykeio/ort and fast encodings via HuggingFace tokenizers, and offers multiple embedding models. The library is part of a multi-language family including Python, Go, and JavaScript versions.

- Supports synchronous usage. No dependency on Tokio. - Uses @pykeio/ort https://github.com/pykeio/ort for performant ONNX inference. - Uses @huggingface/tokenizers https://github.com/huggingface/tokenizers for fast encodings. - Python: fastembed https://github.com/qdrant/fastembed - Go: fastembed-go https://github.com/Anush008/fastembed-go - JavaScript: fastembed-js https://github.com/Anush008/fastembed-js Click to list models - Default BAAI/bge-small-en-v1.5 BAAI/bge-base-en-v1.5 BAAI/bge-large-en-v1.5 BAAI/bge-small-zh-v1.5 BAAI/bge-large-zh-v1.5 BAAI/bge-m3 sentence-transformers/all-MiniLM-L6-v2 sentence-transformers/all-MiniLM-L12-v2 sentence-transformers/all-mpnet-base-v2 sentence-transformers/paraphrase-MiniLM-L12-v2 sentence-transformers/paraphrase-multilingual-mpnet-base-v2 nomic-ai/nomic-embed-text-v1 - pairs with nomic-ai/nomic-embed-text-v1.5 nomic-embed-vision-v1.5 for image-to-text search intfloat/multilingual-e5-small intfloat/multilingual-e5-base intfloat/multilingual-e5-large mixedbread-ai/mxbai-embed-large-v1 Alibaba-NLP/gte-base-en-v1.5 Alibaba-NLP/gte-large-en-v1.5 lightonai/ModernBERT-embed-large - pairs with Qdrant/clip-ViT-B-32-text clip-ViT-B-32-vision for image-to-text search jinaai/jina-embeddings-v2-base-code jinaai/jina-embeddings-v2-base-en google/embeddinggemma-300m - requires nomic-ai/nomic-embed-text-v2-moe nomic-v2-moe feature candle backend - requires Qwen/Qwen3-Embedding-0.6B qwen3 feature candle backend - requires Qwen/Qwen3-Embedding-4B qwen3 feature candle backend - requires Qwen/Qwen3-Embedding-8B qwen3 feature candle backend - requires Qwen/Qwen3-VL-Embedding-2B qwen3 feature candle backend, multimodal via Qwen3VLEmbedding snowflake/snowflake-arctic-embed-xs snowflake/snowflake-arctic-embed-s snowflake/snowflake-arctic-embed-m snowflake/snowflake-arctic-embed-m-long snowflake/snowflake-arctic-embed-l Quantized versions are also available for several models above append Q to the model enum variant, e.g., EmbeddingModel::BGESmallENV15Q . EmbeddingGemma additionally ships a 4-bit build as EmbeddingModel::EmbeddingGemma300MQ4 . Click to list models - Default prithivida/Splade PP en v1 BAAI/bge-m3 Click to list models Click to list models To support the library, please donate to our primary upstream dependency, ort https://github.com/pykeio/ort?tab=readme-ov-file -sponsor-ort - The Rust wrapper for the ONNX runtime. Run the following in your project directory: cargo add fastembed Or add the following line to your Cargo.toml: dependencies fastembed = "5" use fastembed::{TextEmbedding, TextInitOptions, EmbeddingModel}; // With default options let mut model = TextEmbedding::try new Default::default ?; // With custom options let mut model = TextEmbedding::try new TextInitOptions::new EmbeddingModel::AllMiniLML6V2 .with show download progress true .with intra threads 4 , ?; let documents = vec "passage: Hello, World ", "query: Hello, World ", "passage: This is an example passage.", // You can leave out the prefix but it's recommended "fastembed-rs is licensed under Apache 2.0" ; // Generate embeddings with the default batch size, 256 let embeddings = model.embed documents, None ?; println "Embeddings length: {}", embeddings.len ; // - Embeddings length: 4 println "Embedding dimension: {}", embeddings 0 .len ; // - Embedding dimension: 384 use fastembed::{SparseEmbedding, SparseInitOptions, SparseModel, SparseTextEmbedding}; // With default options let mut model = SparseTextEmbedding::try new Default::default ?; // With custom options let mut model = SparseTextEmbedding::try new SparseInitOptions::new SparseModel::SPLADEPPV1 .with show download progress true , ?; let documents = vec "passage: Hello, World ", "query: Hello, World ", "passage: This is an example passage.", "fastembed-rs is licensed under Apache 2.0" ; // Generate embeddings with the default batch size, 256 let embeddings: Vec<SparseEmbedding = model.embed documents, None ?; use fastembed::{ImageEmbedding, ImageInitOptions, ImageEmbeddingModel}; // With default options let mut model = ImageEmbedding::try new Default::default ?; // With custom options let mut model = ImageEmbedding::try new ImageInitOptions::new ImageEmbeddingModel::ClipVitB32 .with show download progress true , ?; let images = vec "assets/image 0.png", "assets/image 1.png" ; // Generate embeddings with the default batch size, 256 let embeddings = model.embed images, None ?; println "Embeddings length: {}", embeddings.len ; // - Embeddings length: 2 println "Embedding dimension: {}", embeddings 0 .len ; // - Embedding dimension: 512 use fastembed::{TextRerank, RerankInitOptions, RerankerModel}; // With default options let mut model = TextRerank::try new Default::default ?; // With custom options let mut model = TextRerank::try new RerankInitOptions::new RerankerModel::BGERerankerBase .with show download progress true , ?; let documents = vec "hi", "The giant panda Ailuropoda melanoleuca , sometimes called a panda bear, is a bear species endemic to China.", "panda is animal", "i dont know", "kind of mammal", ; // Rerank with the default batch size, 256 and return document contents let results = model.rerank "what is panda?", documents, true, None ?; println "Rerank result: {:?}", results ; Alternatively, local model files can be used for inference via the try new from user defined ... methods of respective structs. Helpers in the similarity https://docs.rs/fastembed/latest/fastembed/similarity/ module score and rank the vectors embed returns, so a quick in-memory search needs no extra crate: use fastembed::similarity::{cosine similarity, top k}; // embeddings is the Vec<Embedding from model.embed ... let query = &embeddings 0 ; // Score two vectors directly -1.0, 1.0 , higher = closer let score = cosine similarity query, &embeddings 1 ; // Or rank the corpus: index, score pairs, best first let hits = top k query, &embeddings, 5 ; println "Closest: {:?}", hits ; For larger corpora or persistence, push the vectors to a vector search engine e.g. Qdrant https://qdrant.tech/ and query there. Qwen3 embedding models are available behind the qwen3 feature flag candle backend . dependencies fastembed = { version = "5", features = "qwen3" } js use candle core::{DType, Device}; use fastembed::Qwen3TextEmbedding; let device = Device::Cpu; let model = Qwen3TextEmbedding::from hf "Qwen/Qwen3-Embedding-0.6B", &device, DType::F32, 512, ?; // Text-only usage with the Qwen3-VL embedding checkpoint is also supported: // let model = Qwen3TextEmbedding::from hf "Qwen/Qwen3-VL-Embedding-2B", &device, DType::F32, 512 ?; let embeddings = model.embed & "query: ...", "passage: ..." ?; println "Embeddings length: {}", embeddings.len ; For multimodal text/image usage with Qwen/Qwen3-VL-Embedding-2B : js use candle core::{DType, Device}; use fastembed::Qwen3VLEmbedding; let device = Device::Cpu; let model = Qwen3VLEmbedding::from hf "Qwen/Qwen3-VL-Embedding-2B", &device, DType::F32, 2048, ?; let image embeddings = model.embed images & "tests/assets/image 0.png", "tests/assets/image 1.png" ?; let text embeddings = model.embed texts & "query: blue cat", "query: red cat" ?; println "Image embeddings: {}", image embeddings.len ; println "Text embeddings: {}", text embeddings.len ; The nomic-embed-text-v2-moe https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe model is available behind the nomic-v2-moe feature flag candle backend . First general-purpose MoE embedding model with 100+ language support. dependencies fastembed = { version = "5", features = "nomic-v2-moe" } js use candle core::{DType, Device}; use fastembed::NomicV2MoeTextEmbedding; let device = Device::Cpu; let model = NomicV2MoeTextEmbedding::from hf "nomic-ai/nomic-embed-text-v2-moe", &device, DType::F32, 512, ?; let embeddings = model.embed & "search query: ...", "search document: ..." ?; println "Embeddings length: {}", embeddings.len ; The BGE-M3 model produces dense, sparse, and ColBERT embeddings simultaneously in a single forward pass. use fastembed::{Bgem3Embedding, Bgem3InitOptions, Bgem3Model}; // With default options let mut model = Bgem3Embedding::try new Default::default ?; // With custom options supporting custom max length up to 8192 tokens let mut model = Bgem3Embedding::try new Bgem3InitOptions::new Bgem3Model::BGEM3Q .with max length 1024 .with show download progress true , ?; let documents = vec "Hello, World ", "This is an example passage.", "fastembed-rs is licensed under Apache 2.0", "i dont know" ; // Generate all three representations in a single forward pass let output = model.embed documents, None ?; println "Dense dimension: {}", output.dense 0 .len ; // - Dense dimension: 1024 let sparse emb = &output.sparse 0 ; println "Sparse non-zero tokens: {}", sparse emb.indices.len ; println "ColBERT token count: {}", output.colbert 0 .len ; Note The default quantized model BGEM3Q is optimized for CPUs; passing a GPU execution provider like CUDA will fail. For GPU inference or custom requirements, you can export your own custom model FP32, FP16, or INT8 using the ONNX export script from hf gpahal/bge-m3-onnx-int8 and load it via try new from path . Models download on first use and load from cache afterwards no network needed at runtime once cached . FASTEMBED CACHE DIR — cache location default: .fastembed cache . Equivalent to TextInitOptions::with cache dir . HF HOME — if set, takes precedence over the above. HF ENDPOINT — Hugging Face mirror base URL, for restricted networks. To run models on a GPU via DirectML on Windows, enable the directml feature: dependencies fastembed = { version = "5", features = "directml" } Then pass a DirectML execution provider when initializing a model: use fastembed::{TextEmbedding, TextInitOptions, EmbeddingModel}; use ort::ep::DirectML; let model = TextEmbedding::try new TextInitOptions::new EmbeddingModel::AllMiniLML6V2 .with execution providers vec DirectML::default .into , ?; When DirectML is detected, fastembed automatically disables memory pattern optimization and parallel execution on the ONNX Runtime session, as required by the DirectML execution provider.