A local embedding model, a second smaller model that converts text into vectors, is essential for retrieval-augmented generation (RAG) setups, as it retrieves relevant document chunks for the LLM. Acc…
A developer's investigation into retrieval leaderboards reveals that rankings are highly sensitive to query wording, meaning models topping benchmarks may not perform best on an application's own data…
Researchers introduced SABER-Math, the first fully automated benchmark for evaluating mathematical information retrieval without expert annotation, built from 283K high-school-level math problems. The…
MoganAI released MoganBert-TR, a 149M-parameter Turkish encoder foundation model trained from scratch on 237.3B tokens with a two-stage CLM-to-MLM curriculum, achieving 78.41 on TrGLUE, the best among…
Seahorse, a new open-source memory system for LLM agents, stores persistent, bi-temporal memory in local markdown files readable in Obsidian, addressing the problems of opaque, costly, and unreliable …
Mixedbread AI announced Toast 1, a new open-weight embedding model that reportedly matches or exceeds OpenAI's text-embedding-3-large on standard benchmarks while being significantly cheaper to run. T…
A developer benchmarked a homelab agent memory stack on the LoCoMo benchmark and improved accuracy from 63% to 80% by adding hybrid search with BM25 and a local reranker. The developer found that dens…
Researchers present B1ade, an efficient retrieval-augmented generation (RAG) architecture with a 335M parameter embedding model and a 1B parameter small language model (SLM). B1ade-embed achieves top …
Researchers at arXiv introduce an evaluation-unsupervised benchmark coreset selection method that uses submodular subset selection to pick a small subset of prompts from multiple LLM benchmarks while …
Researchers at The SEMQ Group have developed a new method called Symbolic Embedding Multi-Quantization (SEMQ) that reduces the memory and storage footprint of AI models by decoupling semantic meaning …
A developer identifies the 'index distribution problem' as a fundamental flaw in agent-based search systems, where retrieval indexes encode past relevance judgments rather than semantic truth, causing…
A user requests Hugging Face to add Convence/ParseEmbed as an official benchmark on the Hub. ParseEmbed is a retrieval benchmark for embedding models that tests parse-sensitive meaning under hard nega…
A new analysis of over 5,400 AI models reveals that benchmark scores for large language models are highly correlated, with just five subjects on the MMLU test predicting the remaining 52 with 91% accu…
The article announces the release of six new Sentence Transformers CrossEncoder reranker models, ranging from 17 million to 1 billion parameters, which are built on Ettin ModernBERT encoders and achie…
IBM has released two new multilingual embedding models under the Apache 2.0 license, built on ModernBERT: a compact 97M-parameter model and a full-size 311M-parameter model. Both support over 200 lang…
Public benchmarks like MTEB are unreliable for evaluating embedding models in real-world retrieval systems because their generic, artificially clean data does not reflect actual user behavior or domai…