# Show HN: AI Agents for Osint/Sigint > Source: > Published: 2026-08-29 21:39:05+00:00 # The State of AI Web Scraping in 2026 LAUNCH · AUG.27.2026 · Ritwik 75% of the web is dynamic. Basically database records presented in HTML. Built for humans. For Agents, this problem was solved by: **RAG** was the first generation (2024)- And then, **long-context LLMs** took it to a completely new level (2025) Fast-forward to 2026, and we're basically dumping anything and everything into the context windows; taking Nvidia to the moon. This consists of claws, browser/computer-use, and agentic crawlers, and scrapers. These have made it easier to retrieve data, but the cost has stayed the same. Underneath, these still rely on long-context LLMs. And, here's where they fall short: - Cost at scale: for large datasets, cost grows linearly on a per-page basis. - Hallucinations: LLMs are prone to hallucinations - Context pollution and prompt injection Let me now show you what we have built: **Memoization-driven Knowledge and Data Retrieval** It's an work-in-progress implementation for a continual learning harness focusing on **Browser-use.** ### Large-request correctness versus cost Scatter plot of correctness against average USD cost for Makra, Exa, and Firecrawl. Lower cost is toward the left. Building applications on top of a memoization-engine like [Makra](/) solves all 3 of these problems: - Lower cost at scale: as layouts are memoized, cost comes down to a vector query. - Makra extracts data like a traditional scraper. So, hallucinations? Theoretically, none. - Your agentic loop only has to deal with the DOM nodes it's concerned with: More signal, less noise. We have built an in-house **browser harness** (read-only for now) that does the following for you: - Spins up a browser instance and sets up proxies. - Extracts any structured or tabular data from the web for you. - And you can do all this at 1/10th the cost of all the other previous generations. Try it out in: [/playground](/?playground=true)