Build a Price-Monitoring Agent That Decides, Not Just Logs

A developer published a guide for building a Python price-monitoring agent that uses Decodo's Web Data API to fetch competitor pages and Claude to classify price changes as noise, promotions, or broken pages rather than simply logging them. The build, which uses Python 3.12+, SQLite, Pydantic, selectolax and the Anthropic SDK with no agent framework, also checks prices surfaced in AI answers from ChatGPT and Google AI Mode via the API's /scrape endpoint. The demo targets the public ScrapeMe WooCommerce test store and reaches its first decision within 10 minutes of cloning the repo.

Your price-monitoring script probably logs thousands of prices a day into a dashboard few people open after the first week. Then a competitor cuts prices across a whole category by 20%, and your team might not notice until a week later. What's usually missing is a step that decides whether a change is noise, a promotion, or a broken page. Shoppers also ask ChatGPT and Google AI Mode where to buy, and most scripts don't check the prices in those answers. This guide builds that decision step as an AI agent in Python, and adds a check of the prices in AI answers. The Decodo Web Data API fetches the competitor pages, Claude decides, and the demo reaches its first decision within 10 minutes of cloning the guide's repo. The Decodo Web Data API https://decodo.com/web-data-api gives you these endpoints with a single API key. The documentation https://help.decodo.com/docs/web-data-api-introduction lists the parameters for each one: | Endpoint | What it does | Role in this build | |---|---|---| | /map | Lists URLs from a page and its sitemap in one call | New-SKU discovery | | /crawl | Scrapes a section of a site in one job, as HTML or Markdown | Price tracking, daily | | /scrape | Fetches one page, or asks AI search tools such as ChatGPT and Google AI Mode | New-SKU pages, price rechecks, and prices in AI answers | | /search | Returns live search results | Optional promotion checks | Proxy rotation, JavaScript rendering, and anti-bot handling are built into each endpoint, so your code focuses on prices, not proxies or headless browsers. Every endpoint runs on our proxy network of 125M+ IPs, with a 99.99% success rate. The Web Data API was built for AI agents https://decodo.com/blog/introducing-web-data-api that need live web data. The Decodo dashboard Playground runs requests without writing code. Here it maps ScrapeMe's shop page sorted by newest: Each stage is handled by Decodo, Python, Claude, or you: Python extracts or verifies every price, computes the numbers, and enforces your price floor, the lowest price you allow. Its simple rules also log routine events, such as a price change under 5%, so the agent sees only events that need judgment. The agent works only with these numbers, which Python has already checked. The stack is Python 3.12+, Decodo's Python SDK https://decodo.com/ai/python-sdk , SQLite, Pydantic, selectolax, and the Anthropic SDK, with no agent framework. The target is ScrapeMe https://scrapeme.live/shop/ , a public WooCommerce test store that anyone can use to reproduce the demo. To set up the project: The repo's commands do the work, and each step below explains one: | Command | What it does | Explained in | |---|---|---| | uv run price-agent discover | Finds new products with /map | Step 1 | | uv run price-agent track | Crawls your batches with /crawl , finds what changed, and decides | Steps 2 to 4 | | uv run price-agent eval | Tests the agent's decisions | Step 4 | | uv run price-agent ai-shelf | Checks the prices in AI answers | Step 5 | The code in Steps 1 and 2 shows the Decodo calls behind these commands. You can run it on its own to try an endpoint, but the commands already include it. New SKUs from any section of a store usually appear first on its shop page sorted by newest. Mapping that page finds them, even outside the sections your daily crawl covers. uv run price-agent discover maps the first page of the shop, both in its default order and sorted by newest. For each URL that isn't in the local catalog yet, it fetches the page with /scrape and creates a new product event. The first run only stores the catalog, so new products appear from the second run. Behind the command, one synchronous /map call returns the product URLs, and its path filters https://help.decodo.com/docs/web-data-api-map-introduction keep only product pages. To try it on its own, save this code as try api.py in the repo folder and run uv run --env-file .env python try api.py : python import os from decodo import DecodoClient, DecodoConfig, WebScrapingApiConfig config = DecodoConfig web scraping api=WebScrapingApiConfig api key=os.environ "DECODO API KEY" api = DecodoClient config .web scraping api result = api.map { "url": "https://scrapeme.live/shop/?orderby=date", newest products first "select paths": "^/shop/ ^/ +/?$" , product pages only "exclude paths": "^/shop/page/", "^/cart", "^/my-account", "^/product-tag/" , "geo": "United Kingdom", "locale": "en-GB", } "results" 0 urls = link "url" for link in result "content" "links" print len urls , urls :5 ScrapeMe's shop page sorted by latest lists its newest products first: ScrapeMe's catalog is static. To simulate 2 new arrivals, this run first removes Stakataka and Naganadel, outlined above, from the local catalog: bash $ uv run price-agent discover /map: 32 product URLs on the 2 listing pages, 2 new /scrape: 2 of 2 new URLs had product data 2 change events: {'new product': 2} rule: log only e1 Stakataka: New competitor product with no SKU of ours to compare yet. rule: log only e2 Naganadel: New competitor product with no SKU of ours to compare yet. No event needs judgment. No LLM call. For your own target, set newest listing in config.toml to the store's page sorted by newest. uv run price-agent track crawls the catalog in batches by URL path, such as products whose names start with A or B. It runs one /crawl job per batch and saves every page before the next batch starts. /crawl takes a seed URL plus crawl filters https://help.decodo.com/docs/web-data-api-crawl-introduction and returns 202 Accepted with a crawl id . To try the crawl on its own, add this loop to the end of try api.py and run the file again. It reuses the api client from Step 1: python import time /?$ matches URLs with or without a trailing slash for batch in "^/shop/ A-B ^/ /?$", "^/shop/C ^/ /?$" : crawl id = api.start crawl { "url": "https://scrapeme.live/shop/", "sitemap": "only", take product URLs from the sitemap "select paths": batch , "exclude paths": "^/shop/page/", "^/cart", "^/my-account", "^/product-tag/" , "limit": 100, your page limit for this job "output": "html", html keeps the JSON-LD that Step 3 parses "geo": "United Kingdom", "locale": "en-GB", } "crawl id" while api.get crawl status crawl id "status" == "pending": time.sleep 10 pages = list api.get crawl results crawl id follows every next link print batch, len pages , "pages, first:", pages 0 "url" track also extracts each page's offers and saves them to SQLite right away. These settings matter: You pay only for pages fetched successfully https://help.decodo.com/docs/web-data-api-billing . A page without product data, such as a login page, counts as a missing page, not a removed product. When missing pages pass a threshold you set in config.toml , Python alerts a human. A coverage check compares the URLs /map found with the pages fetched. A freshness check flags a competitor page from our prices.csv if no crawl has fetched it within a time limit you set. After the first run sets the baseline, the agent gets nothing on a day with no changes: bash $ uv run price-agent track /crawl 667c7e31-dafb-4daa-8bc7-bcc80250d768: done, 72 product pages saved, 0 failed /crawl 01c47591-a081-4fae-a215-4c019b28cec9: done, 59 product pages saved, 0 failed total: 131 product pages saved, 0 faulted, 0 without product data coverage: 0 of 131 expected catalog URLs not fetched freshness: 3 of 3 tracked pages seen, oldest 0h ago 0 change events: {} Nothing changed. No LLM call. The repo's GitHub Actions workflow can run /crawl daily and commit prices.db back, because each new runner starts with no price history. For SKUs in volatile eCommerce categories https://decodo.com/blog/most-volatile-ecommerce-categories , add an hourly cron job that crawls only their batches, using the --shards option. A daily crawl of the 2 demo batches, 131 products, comes to about 3.9K pages a month. Price extraction https://decodo.com/blog/price-scraping reads Schema.org Product JSON-LD https://decodo.com/glossary/json-ld first, including product variants, with a CSS selector per store as the fallback. Without any changes, the same code read 200 priced size variants from 19 pages that /crawl fetched from a real Shopify store. The diff stores a missing or zero price as unknown, not as a drop to zero. Because ScrapeMe's prices are static, the repo includes a test fixture. It replays 5 unchanged days from a real /crawl of ScrapeMe, then a 6th day with deliberate test changes. They cover a 3% drop, a category promotion, a stock-out, a misplaced decimal, a missing price, and a new product: bash $ uv run price-agent demo 9 change events: {'price changed': 7, 'stock changed': 1, 'new product': 1} Of the 9 events, 7 are real market changes, and 2 are deliberate store-page errors for the agent to recognize. With just the Decodo key, uv run price-agent demo --no-agent prints the events as JSON. The diff links each competitor URL to your SKU, price, and price floor from our prices.csv . Its core column marks your key value items KVIs , the products shoppers use to judge your prices. Here's the Charizard event the agent receives, showing only the fields it uses to decide: { "id": "e2", "kind": "price changed", "name": "Charizard", "offer key": "https://scrapeme.live/shop/Charizard 8840", "category": "Fire", "our sku": "OUR-006", "core sku": true, "currency": "GBP", "old price": "156.00", "new price": "127.92", "delta pct": -18.0, "our price": "159.00", "gap pct": -19.5, "below floor": true, "history": "156.00", "156.00", "156.00", "156.00", "156.00", "127.92" , "flags": } A negative gap pct means the competitor is cheaper. below floor says whether matching would go below your price floor. flags marks a suspected data problem, such as a store page that briefly shows a wrong price. The agent rechecks a flagged event against the live page first. The agent gets the events that need judgment and a plain-English policy, policy.md , as its system prompt. One rule tells it to match a competitor's price on a KVI when that competitor is more than 5% cheaper. The agent acts only through 4 tools with strict: true schemas, defined in src/price agent/agent.py : php def tool name: str, description: str, properties: dict - dict: return {"name": name, "description": description, "strict": True, "input schema": {"type": "object", "properties": properties, "required": list properties , "additionalProperties": False}} IDS = {"type": "array", "items": {"type": "string"}} TOOLS = tool "log only", "Record events as noise or as a data issue. No one is notified.", {"event ids": IDS, "reason": {"type": "string"}} , tool "alert human", "Notify the pricing team about events that need a person's attention.", {"event ids": IDS, "urgency": {"type": "string", "enum": "low", "normal", "high" }, "reason": {"type": "string"}, "suggested response": {"type": "string"}} , tool "propose strategy change", "Write a pricing proposal for a human to approve. Changes nothing live.", {"event ids": IDS, "our sku": {"type": "string"}, "change type": {"type": "string", "enum": "match price", "hold price", "review rule" }, "proposed price": {"type": "string", "description": "Decimal, e.g. 37.00"}, "reason": {"type": "string"}} , tool "reverify", "Re-fetch one event's page live and re-run extraction before you act on it.", {"event id": {"type": "string"}, "reason": {"type": "string"}} , Python adds the checks a schema can't express. For example, a proposal below your price floor goes back to the agent as a tool error. reverify re-fetches the page through /scrape with the crawl's geo and locale and returns the live price. On the demo's test day, rules logged 2 events, and the agent detected both store-page errors and decided the rest: rule: log only e6 Pikachu: Moved -3.0%, under the 5% noise threshold. rule: log only e9 Mewtwo: New competitor product with no SKU of ours to compare yet. reverify e1 Bulbasaur: snapshot 630.00, live 63.00 reverify e5 Mew: snapshot None, live 136.00 propose strategy change e7 Core SKU: competitor Squirtle went out of stock while cheaper than us gap pct -6.5 , so hold our 139.00 and cancel any planned match. alert human e2, e3, e4, e8 Fire category promotion pattern: 4 SKUs dropped 10% or more in one run Charizard -18.0%, Charmander -18.0%, Growlithe -17.0%, Vulpix -17.0% . Charizard OUR-006, core is 19.5% below us and below floor, and Charmander OUR-004 is also below floor gap -19.7% , so matching would breach the floor. log only e1, e5 Data issues, not market signals: live pages do not reproduce the crawl changes. e1 live price is 63.00 vs the 630.00 snapshot extreme jump , and e5 live price is 136.00 vs the null snapshot price missing or zero . Test the decisions. uv run price-agent eval sends 17 events through the same rules and agent, and compares each decision with an expected decision written by hand. The 17 events are the 9 above, the 2 new products from Step 1, and 6 wrong prices in AI answers. Of these, 4 came from live runs, and 2 are edge cases. Rules decide 6, and the agent decides the other 11: bash model run match reverify cost $ misses claude-haiku-5-5 1 17/17 True 0.0038 claude-haiku-5-5 2 17/17 True 0.0036 claude-haiku-5-5 3 17/17 True 0.0034 claude-opus-5-5 1 17/17 True 0.1396 claude-opus-5-5 2 17/17 True 0.1221 claude-opus-5-5 3 17/17 True 0.1262 In the table, reverify means the agent rechecked every flagged event. With the same 17 events and 3 runs, Opus 5.5 made the same decisions at about 36 times the cost. Haiku is the default. The 17 events make a small, fast regression test. Re-run eval after each policy change. Keep people in control. The agent proposes, and a human approves, so your team stays in control of every live price. Teams usually give the agent more autonomy in stages, starting with small, reversible changes on low-volume SKUs above the price floor. Every alert and proposal includes its evidence for a reviewer to check. In Slack, the promotion alert lists each SKU's price change and price history: This step uses a real product that appears in AI answers, Sony's WH-1000XM6 headphones, and treats Sony's store as your store. Our /scrape templates are ready-made integrations for specific sites, such as the ChatGPT scraper https://decodo.com/scraping/chatgpt and the Amazon scraper https://decodo.com/scraping/ecommerce/amazon . Here they ask ChatGPT, Google AI Mode, and Gemini where to buy the headphones, twice each. Claude Haiku then extracts each retailer's price from the answers. Python keeps only prices that appear in the answer with a currency symbol. It reads each store's live price from product-page JSON-LD or our Amazon and Walmart templates. Each AI price then appears next to the store's live price, and a dash marks a store the answer gave no price for: bash $ uv run price-agent ai-shelf Sony Amazon Best Buy Walmart live 459.99 458.00 458.00 458.00 google ai mode 1 379.99 378.00 398.00 378.00 google ai mode 2 - - 458.00 458.00 chatgpt 1 459.99 - 398.00 359.00 chatgpt 2 - - 378.00 458.00 gemini 1 459.99 - 458.00 - gemini 2 378.00 - 458.00 - 8 of 15 quotes for these 4 stores below the live price, 0 above 1 change events: {'ai misquote': 1} alert human a1 ChatGPT quotes Best Buy's Sony WH-1000XM6 at $378.00, but the live page shows $458, a 17.5% gap. The quote is also below our $459.99. Per policy, the quote is not a buyable price, so we should not match it. Answers change each time you ask. The agent creates an ai misquote event only when every answer from one AI search tool shows a wrong price for a store. Here, only ChatGPT's 2 answers both priced Best Buy too low. Across 7 checks like this one, ChatGPT showed 31 of 44 prices for these stores more than 1% below the store's live price. Google AI Mode did so for 11 of 25. No AI price was more than 1% higher than the live price. ChatGPT's own app, logged out, showed the same pattern. Its Best Buy price, $398.00, matches chatgpt 1 above and is $60 below the live price. The policy matches only live prices, because those are the ones a shopper can buy. When an AI answer shows a wrong price for a competitor, the agent sends a low-urgency alert. It suggests checking the live page before you accept a price-match request based on that AI price. A wrong price of 5% or more for one of your own KVIs is high urgency. That alert tells the team to check the product data your store sends to Google Merchant Center https://support.google.com/merchants/answer/16837055 , which Google AI Mode and Gemini listings use. Check 4 things before you use the agent on a real store: You can extend the pipeline in 3 ways: Price monitoring https://decodo.com/use-cases/price-tracking is most useful when it ends in a decision. Let the Web Data API collect the pages, keep extraction and the diff in Python, and give Claude only the decisions that need judgment. As store pages and AI answers change, check the coverage alerts and re-run eval after each policy change. Start on the Web Data API free plan https://decodo.com/web-data-api/pricing , then clone the repo https://github.com/triposat/price-agent and run uv run price-agent demo to watch the agent decide. Next, set one section of a competitor's store in config.toml , your SKUs and price floors in our prices.csv , and your rules in policy.md .