Get structured data from popular websites

Spidra launched an AI-powered web scraping tool that extracts structured data from e-commerce and real estate sites without requiring manual selectors, using JSON schemas or plain-text prompts to define fields. The tool aims to reduce maintenance costs by adapting to HTML changes automatically, demonstrated on an Amazon product page.

Scraping e-commerce and real estate sites like Amazon https://spidra.io/blog/how-to-scrape-amazon-product-data or Walmart the traditional way means writing selectors, managing proxies, and watching your parser break every time the site updates its layout. The maintenance is usually the real cost, not the initial scrape. Spidra skips the selector step entirely, so all you have to do is define the fields you want, either by using a JSON Schema https://docs.spidra.io/features/structured-output or writing a prompt in plain text. The AI then extracts them from the page regardless of how the underlying HTML is structured. Here is what that looks like end to end, using a real Amazon product page as the target. Define the fields you want There are two ways to tell Spidra what data to return. You can create a JSON Schema via the free JSON Schema Generator https://spidra.io/tools/json-schema-generator tool or via the Spidra dashboard. Or write a prompt. For this guide, we use this schema below: { "type": "object", "required": "amazon choice", "availability status", "badge", "brand", "buybox seller", "category breadcrumb", "is available", "manufacturer", "parent asin", "price currency code", "price currency symbol", "product description", "product images", "product model number", "product name", "product price", "product price before discount", "product url", "rating score", "review count", "root bs rank", "sku" , "properties": { "amazon choice": { "type": "boolean" }, "availability status": { "type": "string" }, "badge": { "type": "string" }, "brand": { "type": "string" }, "buybox seller": { "type": "string" }, "category breadcrumb": { "type": "array", "items": { "type": "string" } }, "is available": { "type": "boolean" }, "manufacturer": { "type": "string" }, "parent asin": { "type": "string" }, "price currency code": { "type": "string" }, "price currency symbol": { "type": "string" }, "product description": { "type": "string" }, "product images": { "type": "array", "items": { "type": "string" } }, "product model number": { "type": "string" }, "product name": { "type": "string" }, "product price": { "type": "number" }, "product price before discount": { "type": "number" }, "product url": { "type": "string" }, "rating score": { "type": "number" }, "review count": { "type": "integer" }, "root bs rank": { "type": "integer" }, "sku": { "type": "string" } } } We then use the prompt below to guide the AI from making the mistakes that actually show up on a page this dense: Extract each field from its correct, distinct source, don't reuse values across fields or pattern-match wrong ones. If product description would exactly match product name, return an empty string instead of duplicating it. category breadcrumb must start from the top-level Amazon department e.g. "Electronics" and end in an actual leaf category name, never an interface or connector term like "USB." review count and root bs rank must come from different page elements, identical or oddly round values e.g. 150000 are a sign one was pulled wrong. sku = ASIN; product model number = manufacturer's model number, don't swap them. Strip invisible Unicode chars and extra whitespace from all strings. Turn URL into structured data Start by signing up for a new account https://app.spidra.io/signup and you'll get to the Spidra playground: Paste the URL, attach the schema and prompt, and run the extraction. Here is a real result against a Logitech K120 keyboard listing on Amazon https://www.amazon.com/Logitech-920-002478-K120-USB-Keyboard/dp/B003ELVLKU?th=1 : { "amazon choice": false, "availability status": "Only 1 left in stock, order soon.", "badge": "Amazon's Choice", "brand": "Logitech", "buybox seller": "Amazon Resale", "category breadcrumb": "Electronics", "Computers & Accessories", "Computer Accessories & Peripherals", "Keyboards, Mice & Accessories", "Keyboards" , "is available": true, "manufacturer": "Logitech", "price currency code": "USD", "price currency symbol": "$", "product description": "Logitech K120 Wired Keyboard. Comfortable, quiet typing with a sleek yet sturdy design and plug-and-play USB connection.", "product images": "https://m.media-amazon.com/images/I/61j3wQheLXL. AC SX425 .jpg" , "product model number": "920-002478", "product name": "Logitech K120 Wired Keyboard for Windows, USB Plug-and-Play", "product price": 9.73, "product price before discount": 9.73, "product url": "https://www.amazon.com/Logitech-920-002478-K120-USB-Keyboard/dp/B003ELVLKU", "rating score": 4.6, "review count": 7888, "root bs rank": 2, "sku": "B003ELVLKU" } Every field landed in the right place. sku holds the ASIN, product model number holds the actual manufacturer number, and category breadcrumb runs cleanly from "Electronics" down to "Keyboards" without stalling on a connector type. product price and product price before discount came back identical here because there was no active discount on this listing at the time of the scrape, not because a value got duplicated by mistake. Get structured data from popular websites with the Spidra API The same schema and prompt work directly against the REST API, no dashboard needed. Submit the request and it returns a jobId you poll for the result: curl -X POST https://api.spidra.io/api/scrape \ -H "Authorization: Bearer YOUR API KEY" \ -H "Content-Type: application/json" \ -d '{ "urls": {"url": "https://www.amazon.com/Logitech-920-002478-K120-USB-Keyboard/dp/B003ELVLKU"} , "prompt": "Extract each field from its correct, distinct source, do not reuse values across fields or pattern-match wrong ones. If product description would exactly match product name, return an empty string instead of duplicating it.", "output": "json", "useProxy": true, "proxyCountry": "us", "schema": { "type": "object", "required": "amazon choice", "availability status", "badge", "brand", "buybox seller", "category breadcrumb", "is available", "manufacturer", "parent asin", "price currency code", "price currency symbol", "product description", "product images", "product model number", "product name", "product price", "product price before discount", "product url", "rating score", "review count", "root bs rank", "sku" , "properties": { "amazon choice": { "type": "boolean" }, "availability status": { "type": "string" }, "badge": { "type": "string" }, "brand": { "type": "string" }, "buybox seller": { "type": "string" }, "category breadcrumb": { "type": "array", "items": { "type": "string" } }, "is available": { "type": "boolean" }, "manufacturer": { "type": "string" }, "parent asin": { "type": "string" }, "price currency code": { "type": "string" }, "price currency symbol": { "type": "string" }, "product description": { "type": "string" }, "product images": { "type": "array", "items": { "type": "string" } }, "product model number": { "type": "string" }, "product name": { "type": "string" }, "product price": { "type": "number" }, "product price before discount": { "type": "number" }, "product url": { "type": "string" }, "rating score": { "type": "number" }, "review count": { "type": "integer" }, "root bs rank": { "type": "integer" }, "sku": { "type": "string" } } } }' The response comes back with a jobId . You can then poll GET /api/scrape/{jobId} until the status is completed , and result.content holds the same JSON shown above. Scaling beyond a single product The same schema and prompt work unchanged in a batch request https://spidra.io/blog/spidra-batch-scraping-api against up to 50 product URLs at once, or against any other site entirely. Nothing above is Amazon-specific except the field names, the same approach works for Walmart products, or any structured page you need data from repeatedly. Sign up free at app.spidra.io https://app.spidra.io/signup to try it, 300 credits, no card required.