{"slug": "aeroloop-the-air-quality-agent-that-keeps-its-own-ghosts", "title": "AeroLoop — The Air-Quality Agent That Keeps Its Own Ghosts", "summary": "A developer built AeroLoop, an open-source self-improving agent that predicts PM2.5 air pollution across London and versions every estimation strategy, preserving failed experiments as 'rejected' ghosts rather than overwriting them. In a simulated 120-pair London day, an interpretable four-parameter linear correction over CAMS model output, tuned by gemma-3-27b-it, cut holdout MAE from 3.94 to 2.99, a roughly 24.1% improvement, while a TabPFN challenger reached 3.08. The results come from a deterministic synthetic dataset, not real pollution readings, and the demo runs locally via Ollama.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine](https://dev.to/challenges/hacktoberfest-week4-2026-10-26)*\n\nWhat if an AI agent remembered every mistake it ever made?\n\nNot just its successful predictions. Not just its latest model weights. Every failed experiment, every rejected idea, every strategy that looked promising but couldn't survive a proper test.\n\nThat's AeroLoop.\n\nAeroLoop is a self-improving AI agent that predicts PM2.5 air pollution across London. It saves every prediction **before** ground truth exists. When the actual reading arrives, the agent scores itself, examines its prediction errors, and attempts to improve its estimation strategy.\n\nBut there's a twist: **it doesn't just learn from its mistakes. It preserves them.**\n\nEvery strategy is versioned. Successful experiments become the new incumbent; unsuccessful ones are saved as `vN-rejected` and displayed on the dashboard as literal ghosts 👻. Nothing is silently overwritten.\n\nThe inspiration came from the idea that an intelligent machine shouldn't just remember what worked. It should remember what failed, why it failed, and what it learned from the experience.\n\nAir pollution is already a ghost story: PM2.5 is the invisible residue of traffic, heating, and weather that lingers in a city long after its source is gone. AeroLoop adds another ghost to the story: the history of an AI learning from its own mistakes.\n\nThe strategy is an intentionally small, interpretable linear correction over the CAMS model output:\n\n```\npred = (\n    pm25_model\n    + humidity_coef * (humidity - 50)\n    + wind_coef * (wind - 5)\n    + precip_coef * precipitation_mm\n    + bias\n)\n```\n\nThe LLM provides the imagination; the holdout provides the judgment.\n\nAny proposed parameters are clamped to sensible bounds. If the model is unavailable or returns invalid output, a deterministic heuristic can propose a candidate instead. The report records exactly which model and parameter source were used. No silent fallbacks.\n\nI ran a simulated London day with 4 stations × 30 hourly ticks, producing 120 matched prediction/reading pairs.\n\nHere are the results:\n\n| Version | Proposed by | What changed | Holdout MAE | Verdict | \n|---|---|---|---|---|\n| v3 | `gemma-3-27b-it` | Bias 2.0 → 3.0 | 3.94 → 3.37 | Kept | \n| v4-tabpfn | TabPFN (Prior Labs) | Foundation model trained on tune split | **3.08** | Recorded as challenger | \n| v5 | `gemma-3-27b-it` | Bias 3.0 → 4.0, humidity → -0.02 | 3.37 → **3.06** | Kept | \n| v6 | `gemma-3-27b-it` | Bias → 4.2, wind → -0.2, precipitation → -0.55 | 3.06 → **2.99** | Kept | \n| v7-rejected | `gemma-3-27b-it` | Bias → 4.5 | 3.01 | **Rejected 👻** | \n\nThe final result:\n\n**Holdout MAE: 3.94 → 2.99 — approximately 24.1% improvement.**\n\nOne particularly interesting subplot: TabPFN initially beat the hand-tuned formula outright. Gemma's incremental tuning subsequently edged ahead. A foundation model and a four-parameter formula competed on the same unseen data, with exactly the same evaluation gate.\n\nAnd when Gemma proposed increasing the bias to 4.5, the holdout said no.\n\nThat strategy became a ghost instead of silently replacing the previous version.\n\n*Important: These results come from a deterministic synthetic demo dataset, not real-world measured pollution readings. The holdout is chronological and was kept separate from strategy tuning.*\n\nTo reproduce the demo:\n\nThe demo can run entirely locally with Ollama.\n\n**GitHub repository:** [https://github.com/udirobert/aeroloop](https://github.com/udirobert/aeroloop)\n\nClone and run:\n\n```\ngit clone https://github.com/udirobert/aeroloop.git\ncd aeroloop/backend\n\npython -m venv .venv\nsource .venv/bin/activate\n\npip install -r requirements.txt\ncp ../.env.example .env\n\nuvicorn main:app --port 8000\n```\n\nThen open the dashboard and run the simulation.\n\nTwo important endpoints:\n\n`POST /demo-day` — generates a complete day of prediction and ground-truth memories using deterministic synthetic data.`POST /self-improve` — runs the complete analyse → propose → holdout-gate → keep/reject loop.\nAeroLoop combines open data, open-weight AI models, local inference, and a persistent agent memory.\n\n**The reasoning engine:**\n\nThe model configuration is switchable, allowing different reasoning engines to compete without changing the core agent architecture.\n\n**The data layer:**\n\n**The self-improvement mechanism:**\n\nThe LLM doesn't directly rewrite the application or modify arbitrary code. It proposes parameters for a small, interpretable estimation formula.\n\nCandidates are validated, bounded, evaluated against the incumbent, and either promoted or rejected.\n\nThis makes the system easier to inspect, reproduce, and trust.\n\nAeroLoop would be a very different project if its intelligence, memory, and evaluation process were locked inside a proprietary API.\n\nOpen innovation made several things possible:\n\n**1. Running with the lights (and internet) off.**\n\nThe reasoning engine can run locally through Ollama using Nemotron Mini. No hosted inference service is required for the local demo.\n\n**2. Freedom to change the brain.**\n\nThe architecture supports different OpenAI-compatible endpoints. Gemma, Nemotron, and other compatible models can be tested without rebuilding the agent.\n\n**3. Memory that belongs to the project.**\n\nPredictions, readings, strategy versions, and rejected experiments live in a local SQLite database. The agent's history isn't trapped in a vendor's infrastructure.\n\n**4. Independent challengers.**\n\nTabPFN can compete against the LLM-guided formula under exactly the same evaluation rules. The system doesn't care which model wins; it cares whether the candidate performs better on unseen data.\n\n**5. Reproducibility and transparency.**\n\nThe data sources, strategy parameters, model choices, fallbacks, and evaluation results are inspectable. If a model fails, the system records what happened rather than pretending everything worked.\n\nOpen innovation made it possible to build an agent whose intelligence can be exchanged, whose memory can be inspected, and whose improvements can be independently challenged.\n\nThe agent's experiment history is preserved in its SQLite database, including the proposals, parameter changes, holdout scores, and rejected versions.\n\nI haven't included a DevRelay session recording for this submission.\n\nThe version history itself demonstrates the agent's decision-making process: every improvement has evidence, and every rejected experiment remains part of the record.\n\nThe most interesting moment wasn't when the agent improved.\n\nIt was when the agent failed.\n\nGemma proposed increasing the bias from 4.2 to 4.5. The candidate looked like another reasonable incremental adjustment. But the holdout evaluation showed that it performed worse.\n\nSo AeroLoop rejected it.\n\nThe strategy was saved as `v7-rejected` and appeared on the dashboard as a ghost.\n\nIt wasn't deleted. It wasn't quietly forgotten. It remained visible as part of the machine's history.\n\nThat is what I wanted the Ghost in the Machine theme to mean.\n\nWe often build AI systems that celebrate their latest answer and hide everything that came before it. AeroLoop takes the opposite approach.\n\nIts ghosts are reminders that intelligence isn't just about getting things right. It's also about remembering when you got things wrong.\n\n**The LLM is the imagination. The holdout is the judge. The graveyard is the audit trail.** 👻\n\n[Add the applicable partner prize categories here once confirmed against the challenge rules.]\n\nBuilt for Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine.\n\nRepository: [https://github.com/udirobert/aeroloop](https://github.com/udirobert/aeroloop)", "url": "https://wpnews.pro/news/aeroloop-the-air-quality-agent-that-keeps-its-own-ghosts", "canonical_source": "https://dev.to/papajams/aeroloop-the-air-quality-agent-that-keeps-its-own-ghosts-o3l", "published_at": "2026-10-31 23:45:34+00:00", "updated_at": "2026-10-31 23:51:52.034080+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "machine-learning", "artificial-intelligence"], "entities": ["AeroLoop", "Gemma", "TabPFN", "Prior Labs", "Ollama", "London", "GitHub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/aeroloop-the-air-quality-agent-that-keeps-its-own-ghosts", "markdown": "https://wpnews.pro/news/aeroloop-the-air-quality-agent-that-keeps-its-own-ghosts.md", "text": "https://wpnews.pro/news/aeroloop-the-air-quality-agent-that-keeps-its-own-ghosts.txt", "jsonld": "https://wpnews.pro/news/aeroloop-the-air-quality-agent-that-keeps-its-own-ghosts.jsonld"}}