{"slug": "get-structured-data-from-popular-websites", "title": "Get structured data from popular websites", "summary": "Spidra launched an AI-powered web scraping tool that extracts structured data from e-commerce and real estate sites without requiring manual selectors, using JSON schemas or plain-text prompts to define fields. The tool aims to reduce maintenance costs by adapting to HTML changes automatically, demonstrated on an Amazon product page.", "body_md": "Scraping e-commerce and real estate sites like [Amazon](https://spidra.io/blog/how-to-scrape-amazon-product-data) or Walmart the traditional way means writing selectors, managing proxies, and watching your parser break every time the site updates its layout. The maintenance is usually the real cost, not the initial scrape.\n\nSpidra skips the selector step entirely, so all you have to do is define the fields you want, either by using a [JSON Schema](https://docs.spidra.io/features/structured-output) or writing a prompt in plain text. The AI then extracts them from the page regardless of how the underlying HTML is structured.\n\nHere is what that looks like end to end, using a real Amazon product page as the target.\n\n## Define the fields you want\n\nThere are two ways to tell Spidra what data to return. You can create a JSON Schema via the free [JSON Schema Generator](https://spidra.io/tools/json-schema-generator) tool or via the Spidra dashboard.\n\nOr write a prompt. For this guide, we use this schema below:\n\n```\n{\n  \"type\": \"object\",\n  \"required\": [\n    \"amazon_choice\", \"availability_status\", \"badge\", \"brand\",\n    \"buybox_seller\", \"category_breadcrumb\", \"is_available\",\n    \"manufacturer\", \"parent_asin\", \"price_currency_code\",\n    \"price_currency_symbol\", \"product_description\", \"product_images\",\n    \"product_model_number\", \"product_name\", \"product_price\",\n    \"product_price_before_discount\", \"product_url\", \"rating_score\",\n    \"review_count\", \"root_bs_rank\", \"sku\"\n  ],\n  \"properties\": {\n    \"amazon_choice\":                  { \"type\": \"boolean\" },\n    \"availability_status\":            { \"type\": \"string\" },\n    \"badge\":                          { \"type\": \"string\" },\n    \"brand\":                          { \"type\": \"string\" },\n    \"buybox_seller\":                  { \"type\": \"string\" },\n    \"category_breadcrumb\":            { \"type\": \"array\", \"items\": { \"type\": \"string\" } },\n    \"is_available\":                   { \"type\": \"boolean\" },\n    \"manufacturer\":                   { \"type\": \"string\" },\n    \"parent_asin\":                    { \"type\": \"string\" },\n    \"price_currency_code\":            { \"type\": \"string\" },\n    \"price_currency_symbol\":          { \"type\": \"string\" },\n    \"product_description\":            { \"type\": \"string\" },\n    \"product_images\":                 { \"type\": \"array\", \"items\": { \"type\": \"string\" } },\n    \"product_model_number\":           { \"type\": \"string\" },\n    \"product_name\":                   { \"type\": \"string\" },\n    \"product_price\":                  { \"type\": \"number\" },\n    \"product_price_before_discount\":  { \"type\": \"number\" },\n    \"product_url\":                    { \"type\": \"string\" },\n    \"rating_score\":                   { \"type\": \"number\" },\n    \"review_count\":                   { \"type\": \"integer\" },\n    \"root_bs_rank\":                   { \"type\": \"integer\" },\n    \"sku\":                            { \"type\": \"string\" }\n  }\n}\n```\n\nWe then use the prompt below to guide the AI from making the mistakes that actually show up on a page this dense:\n\n```\nExtract each field from its correct, distinct source, don't reuse values across fields or pattern-match wrong ones. If product_description would exactly match product_name, return an empty string instead of duplicating it. category_breadcrumb must start from the top-level Amazon department (e.g. \"Electronics\") and end in an actual leaf category name, never an interface or connector term like \"USB.\" review_count and root_bs_rank must come from different page elements, identical or oddly round values (e.g. 150000) are a sign one was pulled wrong. sku = ASIN; product_model_number = manufacturer's model number, don't swap them. Strip invisible Unicode chars and extra whitespace from all strings.\n```\n\n## Turn URL into structured data\n\nStart by [signing up for a new account](https://app.spidra.io/signup) and you'll get to the Spidra playground:\n\nPaste the URL, attach the schema and prompt, and run the extraction. Here is a real result against a [Logitech K120 keyboard listing on Amazon](https://www.amazon.com/Logitech-920-002478-K120-USB-Keyboard/dp/B003ELVLKU?th=1):\n\n```\n{\n  \"amazon_choice\": false,\n  \"availability_status\": \"Only 1 left in stock, order soon.\",\n  \"badge\": \"Amazon's Choice\",\n  \"brand\": \"Logitech\",\n  \"buybox_seller\": \"Amazon Resale\",\n  \"category_breadcrumb\": [\n    \"Electronics\",\n    \"Computers & Accessories\",\n    \"Computer Accessories & Peripherals\",\n    \"Keyboards, Mice & Accessories\",\n    \"Keyboards\"\n  ],\n  \"is_available\": true,\n  \"manufacturer\": \"Logitech\",\n  \"price_currency_code\": \"USD\",\n  \"price_currency_symbol\": \"$\",\n  \"product_description\": \"Logitech K120 Wired Keyboard. Comfortable, quiet typing with a sleek yet sturdy design and plug-and-play USB connection.\",\n  \"product_images\": [\n    \"https://m.media-amazon.com/images/I/61j3wQheLXL._AC_SX425_.jpg\"\n  ],\n  \"product_model_number\": \"920-002478\",\n  \"product_name\": \"Logitech K120 Wired Keyboard for Windows, USB Plug-and-Play\",\n  \"product_price\": 9.73,\n  \"product_price_before_discount\": 9.73,\n  \"product_url\": \"https://www.amazon.com/Logitech-920-002478-K120-USB-Keyboard/dp/B003ELVLKU\",\n  \"rating_score\": 4.6,\n  \"review_count\": 7888,\n  \"root_bs_rank\": 2,\n  \"sku\": \"B003ELVLKU\"\n}\n```\n\nEvery field landed in the right place. `sku`\n\nholds the ASIN, `product_model_number`\n\nholds the actual manufacturer number, and `category_breadcrumb`\n\nruns cleanly from \"Electronics\" down to \"Keyboards\" without stalling on a connector type. `product_price`\n\nand `product_price_before_discount`\n\ncame back identical here because there was no active discount on this listing at the time of the scrape, not because a value got duplicated by mistake.\n\n## Get structured data from popular websites with the Spidra API\n\nThe same schema and prompt work directly against the REST API, no dashboard needed. Submit the request and it returns a `jobId`\n\nyou poll for the result:\n\n```\ncurl -X POST https://api.spidra.io/api/scrape \\\n  -H \"Authorization: Bearer YOUR_API_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"urls\": [{\"url\": \"https://www.amazon.com/Logitech-920-002478-K120-USB-Keyboard/dp/B003ELVLKU\"}],\n    \"prompt\": \"Extract each field from its correct, distinct source, do not reuse values across fields or pattern-match wrong ones. If product_description would exactly match product_name, return an empty string instead of duplicating it.\",\n    \"output\": \"json\",\n    \"useProxy\": true,\n    \"proxyCountry\": \"us\",\n    \"schema\": {\n      \"type\": \"object\",\n      \"required\": [\n        \"amazon_choice\", \"availability_status\", \"badge\", \"brand\",\n        \"buybox_seller\", \"category_breadcrumb\", \"is_available\",\n        \"manufacturer\", \"parent_asin\", \"price_currency_code\",\n        \"price_currency_symbol\", \"product_description\", \"product_images\",\n        \"product_model_number\", \"product_name\", \"product_price\",\n        \"product_price_before_discount\", \"product_url\", \"rating_score\",\n        \"review_count\", \"root_bs_rank\", \"sku\"\n      ],\n      \"properties\": {\n        \"amazon_choice\":                  { \"type\": \"boolean\" },\n        \"availability_status\":            { \"type\": \"string\" },\n        \"badge\":                          { \"type\": \"string\" },\n        \"brand\":                          { \"type\": \"string\" },\n        \"buybox_seller\":                  { \"type\": \"string\" },\n        \"category_breadcrumb\":            { \"type\": \"array\", \"items\": { \"type\": \"string\" } },\n        \"is_available\":                   { \"type\": \"boolean\" },\n        \"manufacturer\":                   { \"type\": \"string\" },\n        \"parent_asin\":                    { \"type\": \"string\" },\n        \"price_currency_code\":            { \"type\": \"string\" },\n        \"price_currency_symbol\":          { \"type\": \"string\" },\n        \"product_description\":            { \"type\": \"string\" },\n        \"product_images\":                 { \"type\": \"array\", \"items\": { \"type\": \"string\" } },\n        \"product_model_number\":           { \"type\": \"string\" },\n        \"product_name\":                   { \"type\": \"string\" },\n        \"product_price\":                  { \"type\": \"number\" },\n        \"product_price_before_discount\":  { \"type\": \"number\" },\n        \"product_url\":                    { \"type\": \"string\" },\n        \"rating_score\":                   { \"type\": \"number\" },\n        \"review_count\":                   { \"type\": \"integer\" },\n        \"root_bs_rank\":                   { \"type\": \"integer\" },\n        \"sku\":                            { \"type\": \"string\" }\n      }\n    }\n  }'\n```\n\nThe response comes back with a `jobId`\n\n. You can then poll `GET /api/scrape/{jobId}`\n\nuntil the status is `completed`\n\n, and `result.content`\n\nholds the same JSON shown above.\n\n## Scaling beyond a single product\n\nThe same schema and prompt work unchanged in a [batch request](https://spidra.io/blog/spidra-batch-scraping-api) against up to 50 product URLs at once, or against any other site entirely.\n\nNothing above is Amazon-specific except the field names, the same approach works for Walmart products, or any structured page you need data from repeatedly.\n\nSign up free at [app.spidra.io](https://app.spidra.io/signup) to try it, 300 credits, no card required.", "url": "https://wpnews.pro/news/get-structured-data-from-popular-websites", "canonical_source": "https://spidra.io/blog/scrape-structured-data", "published_at": "2026-07-08 00:00:00+00:00", "updated_at": "2026-07-08 18:15:54.006828+00:00", "lang": "en", "topics": ["ai-tools", "artificial-intelligence"], "entities": ["Spidra", "Amazon", "Walmart", "Logitech"], "alternates": {"html": "https://wpnews.pro/news/get-structured-data-from-popular-websites", "markdown": "https://wpnews.pro/news/get-structured-data-from-popular-websites.md", "text": "https://wpnews.pro/news/get-structured-data-from-popular-websites.txt", "jsonld": "https://wpnews.pro/news/get-structured-data-from-popular-websites.jsonld"}}