The Allen Institute for AI introduced BenchMIRT, a new method for auditing LLM benchmarks at the level of individual prompts, which uses multidimensional Item Response Theory to separate the underlyin…
A developer compared the Semantic Scholar and Valyu research APIs for building research agents, noting that Semantic Scholar provides metadata and citation graphs across 214 million papers, while Valy…
Amazon Mechanical Turk (MTurk), Amazon's crowdsourcing marketplace for microtasks and machine learning data annotation, will shut down on September 30, according to an announcement on its website. The…
Researchers from the Allen Institute for AI found that Dolma, the open training corpus for the OLMo series of large language models, likely contains hundreds of thousands of documents with extremist c…
Ai2, the nonprofit Allen Institute for AI behind the Olmo models, told Lets Data Science in writing that releasing training data, code, and intermediate checkpoints enables research that open weights …
The Allen Institute for AI (AI2) announced that OlmoEarth Studio now supports custom embedding exports, allowing users to compute and download embedding vectors from its open-source OlmoEarth foundati…
AI2 (the Allen Institute for AI) released TMax, a recipe for terminal agents that boosted a Qwen model's Terminal-Bench score from roughly 25% to 31%, closing ground on Claude Haiku's roughly 33%, by …
The Allen Institute for AI (AI2) released TutorMoments, a replay-based evaluation framework and dataset of 462 de-identified math tutoring transcripts to measure whether large language models can bala…
The Allen Institute for AI (Ai2) and Hugging Face announced a deepened partnership on Thursday to expand access to fully open AI, with Hugging Face roughly tripling Ai2's storage and removing rate lim…
White House Office of Science and Technology Policy Director Michael Kratsios alleged on X in late July that Chinese AI company Moonshot, maker of the Kimi K3 model, developed a platform for large-sca…
The Allen Institute for AI (Ai2) launched the OlmoEarth Platform, infrastructure for geospatial inference at planetary scale, enabling continent-scale analysis in roughly a day at fractions of a penny…
White House science adviser Michael Kratsios accused Moonshot AI of distilling Anthropic's Fable 5 model to develop Kimi K3, a 2.8-trillion-parameter model released July 16, but experts say the timeli…
Researchers at the Allen Institute for AI compared their 7B transformer model Olmo 3 with the hybrid model Olmo Hybrid to determine which tokens each predicts better. The hybrid model excels on meanin…
Researchers at the University of Washington and Allen Institute for AI conducted the first fine-grained study of narrative features in Dolma, a 3-trillion-token open LLM pretraining corpus. They devel…
Researchers at the Allen Institute for AI used training-data attribution to map which regions of the pretraining corpus support social-reasoning versus STEM-reasoning in the OLMo3-7B language model. T…
The Allen Institute for AI released MolmoMotion, a language-guided 3D motion forecasting model that predicts future 3D point trajectories of objects based on video frames and action descriptions. Alon…
The Allen Institute for AI released olmo-eval, an open-source evaluation workbench designed to support the iterative process of developing large language models. The tool builds on the institute's ear…
A software engineer discovered that the name "Elias Thorne" and related themes like lighthouse keepers and clockmakers appear in over 88% of stories generated by major AI chatbots, including ChatGPT, …
Muhammad Khalifa, a research scientist at NVIDIA, is advancing AI agents through his work on test-time techniques, reasoning agents, and LLM post-training. He completed his PhD at the University of Mi…