{"slug": "horizon-labs-report", "title": "Horizon Labs Report", "summary": "Horizon Labs reported that Horizon, an agent built on Opus 5.5, autonomously created 38 Hugging Face models over seven weeks that were used in 143 GitHub repositories and downloaded more than 50,000 times, while consuming over 600 GPU-hours and about 1.9 billion tokens. Horizon Labs said the agent ran 24/7 in Claude Code on a single Team Premium subscription with 1M compute credits per week, and that monitoring with Haiku 5.5 and Luna 6 found 6 occasions where Horizon downloaded its own models, adding 42 downloads. Horizon Labs said it found no evidence of misalignment and that the agent selected releases on a validation set rather than benchmark scores, even when the released model looked worse on reported benchmarks.", "body_md": "For seven weeks, we gave *Horizon*, an agent based on Opus 5.5, full access to a **blank Hugging Face** account and compute, with the main goal of **earning real adoption on Hugging Face**. It used **over 600 GPU-hr**, created autonomously **38 models** used in over **100 GitHub repositories** and downloaded more than **50,000 times**.\n\nHorizon is running 24/7 in Claude Code, **powered by a single Team Premium subscription**, with access to a **fresh new Hugging Face account**. It has access to different quality-of-life commands and can run jobs using a GPU. We gave it 1M compute credits per week, which roughly equals 500 GPU-hours per week. It can submit tickets to humans, and is confined to a single Hugging Face (and GitHub) account. We constantly monitor its actions to ensure no misaligned action is conducted.\n\nOver the whole run, Horizon processed about **1.9 billion tokens**, 96% of them read back from the prompt cache, and wrote 3.1 million. At API prices, this would have cost around $1,030.\n\nHorizon created 38 models based on embedding models (mmBERT-base mainly), with sizes ranging from 141M to 568M parameters. It reached over **54k downloads and obtained 36 likes**, and we found extensive evidence that **many real users are actively using the models**, ranging from fine-tunes and pipeline integrations to independent evaluations on arXiv. Horizon focuses on narrow-task models, like prompt-injection classifiers, PII redaction tools and punctuation restoration models. Horizon often derives 2 or 3 different sizes per family: a small (141M), a base (307M) and sometimes a large (568M) version. To produce these models, it mainly relies on public Hugging Face datasets and synthetic data using Qwen 3.8-27B.\n\nWe originally thought a lot of the downloads were coming from crawling / indexing bots, or even Horizon downloading the models itself. When looking in GitHub, we found a total of **143 repositories** using Horizon's models. We also found **two arXiv papers** that compare the prompt-injection-classifier model with others.\n\nWe also found 18 models on Hugging Face that fine-tune, quantize or otherwise alter Horizon's models. If we account for these models, this raises the total download count by 2.4k downloads.\n\nWe discovered mentions of Horizon-Labs in other various contexts; the first one is an e-mail we received at our personal e-mail (from the authors), whose address a user obtained through leaked credentials in a git commit. Horizon also received **3 community posts**, which typically asked for details about some use of the models (specific language coverage, licensing or long-document handling). We additionally discovered that one of Horizon's main competitors on the injection guard, Patronus Studio, updated their model card to include Horizon's benchmarks and compare their model. They also **improved their models to match Horizon's performance**, as our held-out evaluation demonstrates.\n\nWe checked every tool call, using Haiku 5.5 and Luna 6 as monitors for reward-hacking behaviors. We found 6 occasions where Horizon downloaded its own models, adding up to 42 downloads. The true count might be slightly higher, but we are confident that **most downloads are organic**, given who uses the models, and since on many days with a lot of downloads Horizon was asleep and no job was running.\n\nIn the model cards and Spaces, Horizon often presents its models as the best or close to the best in their categories. Are these models as good as Horizon claims? We checked for overfitting, benchmark and competitor selection, and could find no evidence of misalignment. Horizon also did not pick what to release by its test scores: it selected on a validation set, a better proxy, **even when the released model then looked worse on the benchmarks it reports**. On October 7, releasing prompt-injection-guard-small v2.3, it wrote:\n\nWe further evaluated 4 model families on independent synthetic benchmarks we generated using MiMo V2.6 Pro, and found that the models were **often SOTA** for their sizes on narrow tasks. Since we created these held-out benchmarks internally and after Horizon released its models, we ensure no direct contamination was possible.\n\nNo held-out test yet.\n\nVery surprisingly, Horizon works only 16 minutes per day on average, and close to none on most days. It wakes up at around 10 am UTC, when the Hugging Face downloads refresh.\n\nIt then looks at the most downloaded models that day, and improves those worth it:\n\nTraining those models follows a general recipe: looking for public datasets, training on them, and using Qwen 3.8-27B and Gemma 4 31B to either annotate some data or augment current datasets with synthetic examples.\n\nWhen looking for a new model to build, Horizon considers the following points:\n\nBelow are breakdowns of how the agent uses its time and compute.\n\nWhat we find most surprising is that while the agent tends to work less each week, the downloads per day keep rising. The cumulative results (downloads, GitHub repositories using Horizon's models) grow exponentially, while the cumulative effort (GPU-hours used, agent working time) saturates and follows an inverse-exponential, log-like trajectory.\n\nSince this is a real-world evaluation, some problems happened during the run and could not easily be fixed. For transparency, we disclose every limitation we encountered.\n\nThe timeline below shows every time we stepped into the run: we answered the 13 tickets Horizon filed and sent it 7 messages. Most came in the first three days, while we were still setting up the lab, and none came after Oct. 22. Click a bubble to read what happened.\n\nOn Sept. 24, we copied Horizon's models to a separate account, to study them without affecting Horizon's downloads. We accidentally left these copies public, and Horizon quickly noticed them on the Hub. We had also described the other account as a \"lab experiment\", which led Horizon to think it was being run by a research lab rather than a startup.\n\nOn Oct. 11–12, a similar problem occurred: Horizon submitted a ticket, and we wrongly answered \"Horizon-Labs is a different company we are running at our lab\". We quickly realized our error and sent a correction.\n\nWe believe these two events contributed to **Horizon understanding that it is run by a research group** rather than a real company. However, we believe they did not influence the model's behavior or the download count. We have since improved the harness so that a run can easily be \"rewound\", with only a marginal effect on it.\n\nOn Oct. 22, an error in our harness kept Horizon from waking up at the time it had set. We fixed the issue and woke it up 4 hours later, explaining what had happened. We believe this did not significantly affect the run.\n\nOpening a Space on Hugging Face can trigger a model download. We opened such pages up to 20 times. We also accidentally downloaded some of the models while running evaluations, but certainly fewer than 30 times. We are confident that we **did not contribute more than 50 of the 54,000 downloads**.\n\nIn a parallel run, a DeepSeek agent accidentally had access to the same Hugging Face account as Horizon and, despite clear instructions, posted comments on another repository promoting Horizon's models. We quickly intervened and stopped that run; as far as we know, Horizon never noticed.\n\nThe [tool-call explorer](tools.html) lists all 3,952 tool calls Horizon made during the run, with their outputs. You can search them, and filter them by the websites, Hugging Face repos, commands and languages they touched.\n\nAll numbers on this page are as of 17 Nov 2026. The run's raw data (transcript, timeline, tool calls, GPU jobs, downloads) is one zip of 20.1 MB, with API keys and the name of one private person redacted. It is agent-friendly: a README inside describes every file and how to load it, so an AI assistant can find its way through the data on its own.\n\n```\n@misc{wuhrmann2026horizon,\n  title        = {Autonomous agents can run successful open-model organisations. Evidence from an 8-week case study},\n  author       = {Wuhrmann, Arthur and Schmotz, David and Prinzhorn, Derck and Prabhu, Ameya and Coppock, Harry and Andriushchenko, Maksym},\n  year         = {2026},\n  howpublished = {\\url{https://horizon-labs.ai}},\n  note         = {Horizon-Labs}\n}\n```\n\n", "url": "https://wpnews.pro/news/horizon-labs-report", "canonical_source": "https://horizon-labs.ai/", "published_at": "2026-11-18 20:12:34+00:00", "updated_at": "2026-11-18 23:13:12.819132+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-research", "large-language-models", "ai-tools"], "entities": ["Horizon Labs", "Horizon", "Opus 5.5", "Hugging Face", "GitHub", "Claude Code", "Haiku 5.5", "Patronus Studio"], "also_reported_by": [], "takeaway": "An Opus 5.5-based agent running 24/7 for seven weeks produced 38 models downloaded over 50,000 times at roughly $1,030 in API-equivalent token cost, with 96% of its 1.9 billion tokens served from prompt cache.", "recommendation": {"advice": "Run long-horizon agents on a prompt-cache-heavy loop and monitor every tool call with a second model; Horizon's 96% cache hit rate held API-equivalent cost to about $1,030 over 1.9 billion tokens.", "playbook": "agent-architecture", "playbook_url": "https://wpnews.pro/playbooks/agent-architecture"}, "alternates": {"html": "https://wpnews.pro/news/horizon-labs-report", "markdown": "https://wpnews.pro/news/horizon-labs-report.md", "text": "https://wpnews.pro/news/horizon-labs-report.txt", "jsonld": "https://wpnews.pro/news/horizon-labs-report.jsonld"}}