Cost doesn't set price: how I got a product decision wrong 4 times in one day

A developer spent a day using an AI agent to research a product decision and saw its recommendation reverse four times, all due to estimating selling price from cost rather than market data. The agent's errors were compounded by a US import rule change and a data collection bug, leading the developer to conclude that cost does not set price—the market does. The developer now uses public data sources like Shopify's /products.json to measure actual competitor prices.

I spent a day letting an AI agent research a product decision for me. It changed its recommendation four times. Every reversal was caused by the same mistake, and I didn't see it until the fourth one. The mistake is embarrassingly basic: I was estimating the selling price from the cost. Here's how that plays out when you're moving fast, and the three data sources that fixed it — all free, all public. I was screening physical products to advertise. Two questions matter: I built a gate for question 1 first: scrape the ad library, count how many distinct sellers have been running ads for 30+ days, filter for keyword relevance. Ads that run for a year are a good proxy for "this is profitable for someone." That gate worked. Then I needed selling prices for question 2, and I didn't have them. So I did the natural thing: took the supplier cost and multiplied by a typical retail markup. That single shortcut caused every reversal below. First recommendation: run the test in the US and UK. Then I checked import rules properly. The US de minimis exemption — which let sub-$800 parcels enter duty-free — ended for all countries in August 2025. Small parcels from China now pay duty plus a flat customs entry fee of roughly $17 per parcel. That flat fee is the killer, and it took me a while to see why: | Product | Goods cost | Duty 37.5% | Flat entry fee | Fee as % of goods | |---|---|---|---|---| | A | $4.95 | $1.86 | $17.00 | 344% | | B | $14.09 | $5.28 | $17.00 | 121% | | C | $24.80 | $9.30 | $17.00 | 69% | The percentage duty is small — it's charged on the cheap goods value. The flat fee doesn't care what the item costs , so it lands hardest on exactly the low-price items that direct-to-consumer import was built on. The UK still has its £135 relief until October 2028 . So: UK, not US. Fine. Real finding. But notice it's about landed cost — still the cost side. Second recommendation: three specific products, ranked. Except two of them had been scanned before I'd deployed a relevance filter on the ad scraper. Without it, the scraper counted every ad the keyword search returned — including romance novels and migraine coaches that happened to contain the word "ice" or "face." I rescanned with the filter on. Two of my top picks turned out to be red oceans 3,000 and 6,500 active ads . The "sellers" in the old data included what were obviously personal accounts and coupon-spam pages, not brands. Lesson, cheap version: when you fix a data collection bug, the old rows don't fix themselves. Write the backfill. I now have a rescan --stale mode that finds every row collected before the fix and re-runs it. Third recommendation, with real conviction this time: one product, clean on both gates, priced at $102, break-even ROAS under 2. Then I went to look at what competitors actually charge — and the leading brand in that category sells the equivalent product for $32.99 . I had computed $102 from supplier cost × 3 . There is no world in which that product sells for $102. At $33, after VAT and payment fees, the ad budget per order is about four dollars. It's not a marginal business, it's an impossible one. That was reversal four, and it finally made the pattern visible. Cost does not set price. The market sets price. Cost only tells you what's left. Look at what my markup assumption implied versus reality: | Product | Supplier cost | My estimate ×3 | What the leader actually charges | Real multiple | |---|---|---|---|---| | Sleep mask | $6.95 | $21 | $127 catalog median | 18× | | Bidet attachment | $34 | $102 | $33 | 1× | One product supports an 18× markup. Another supports 1×. Same formula, opposite errors — and no amount of care about the cost side would have caught either. The fix isn't a better multiplier. It's to stop estimating a number you can go and measure. /products.json is public Most DTC brands run Shopify, and Shopify exposes the full catalog as JSON with no auth: https://<domain /products.json?limit=250 You get every product, every variant, every price: python import json, urllib.request, statistics req = urllib.request.Request f"https://{domain}/products.json?limit=250", headers={"User-Agent": "Mozilla/5.0"} products = json.load urllib.request.urlopen req "products" prices = float v "price" for p in products for v in p.get "variants", if v.get "price" and float v "price" 0 print len products , "products, median", statistics.median prices That five-line query is what ended reversal 4. It's also what showed me something I'd have missed entirely: the category leader has 95 SKUs , and most of them are consumables — dryer sheets, cleaning solution, refills. They're not selling one gadget. They're selling a razor-and-blades business, which is why they can afford customer acquisition I can't. From the ad copy. Advertisers put their own URL in the ad text, so a regex over scraped ad bodies gives you the competitor list for free: DOMAIN = re.compile r"\b ?: a-z0-9 a-z0-9- \. + ?:com|co\.uk|shop|store \b", re.I Filter out the obvious noise facebook.com , amazon.com , your ad platform and rank by frequency — an advertiser's own domain repeats across their ads. Free-tier traffic estimators 3 lookups/day is plenty give month-over-month change and channel mix. Two competitors, same day: | | Leader A | Leader B | |---|---|---| | Monthly visits | ~232K | ~52K | | MoM change | +38.6% | −10.9% | | Top channel | Paid social 32% | Paid search 37% | | Bounce / pages / duration | 42.9% / 3.18 / 2m07s | 74.8% / 1.82 / 31s | Two things fell out of that table that nothing else told me: The screening gate now refuses to be confident on estimated prices: if retail price is None and score = PASS THRESHOLD: verdict = "WATCH" never PASS on an estimate reasons.append "price is estimated, not measured — go fetch the real one" That one rule would have prevented three of the four reversals. Not because the estimate was badly calibrated, but because an estimate and a measurement should never be allowed to look the same downstream. Once $102 estimated becomes just 102 in a table, every decision after it inherits a confidence the number never had. If you're building anything that ranks options, tag every input with how you got it. The ones you guessed are the ones that will move. I write about running unattended automation and AI agents in production, including the parts where the agent — or I — confidently get it wrong. The full system I use for this is the Claude Code Automation Playbook https://alphatech4.gumroad.com/l/claude-code-automation-playbook .