{"slug": "scraping-dynamic-web-pages-without-selectors-using-ai-vision-typescript-tutorial", "title": "Scraping Dynamic Web Pages Without Selectors Using AI Vision (TypeScript/JavaScript Tutorial)", "summary": "A developer built a selector-free web scraper using Opticparse, an AI-powered tool that captures webpage screenshots and uses Gemini's multimodal vision to extract structured JSON data. The scraper bypasses dynamic CSS class names, shadow DOMs, and obfuscated HTML by mimicking human visual perception. The official Opticparse JavaScript/TypeScript SDK enables data extraction in under 10 lines of code.", "body_md": "****# Scraping Dynamic Web Pages Without Selectors Using AI Vision (TypeScript/JavaScript Tutorial)\n\nWeb scraping has traditionally been a game of cat-and-mouse. You spend hours writing fine-tuned CSS selectors or XPath paths, only for the website to change its layout or class names (especially on modern frameworks with generated CSS class names like `css-1ux802d`\n\n), breaking your entire data pipeline overnight.\n\nIn this tutorial, we will learn how to build a **selector-free scraper** using **Opticparse**, an AI-powered scraping tool that captures webpage screenshots and uses Gemini's multimodal vision intelligence to extract structured JSON data.\n\nWe will use the official **Opticparse JavaScript/TypeScript SDK** to extract data in less than 10 lines of code.\n\nInstead of parsing HTML source code directly, Opticparse:\n\nBecause it mimics how a real human looks at the page, it does not care about dynamic CSS class name changes, shadow DOMs, or obfuscated HTML.\n\nInstall the official client library:\n\n```\nnpm install opticparse-js\n```\n\nYou can get an API key in two ways:\n\n`OPTICPARSE_API_KEY`\n\n.Let's say we want to scrape the top 5 articles, their link URLs, and score points from the homepage of Hacker News.\n\nHere is how you do it:\n\n``` js\nimport { OpticparseClient } from 'opticparse-js';\n\n// Initialize the client. \n// If using the RapidAPI marketplace, set useRapidApi: true\nconst client = new OpticparseClient({\n  apiKey: 'YOUR_RAPIDAPI_KEY_HERE',\n  useRapidApi: true\n});\n\nasync function runScrape() {\n  console.log('Scraping Hacker News articles...');\n\n  try {\n    const data = await client.scrape({\n      targetUrl: 'https://news.ycombinator.com',\n      extractionQuery: 'Extract the top 5 article titles, their link URLs, and score points as a JSON list of objects.',\n      viewportWidth: 1280,\n      viewportHeight: 1000\n    });\n\n    console.log('Scraped Data Output:');\n    console.log(JSON.stringify(data, null, 2));\n\n  } catch (error) {\n    console.error('Scraping failed:', error);\n  }\n}\n\nrunScrape();\n```\n\nThe client will automatically handle the asynchronous execution, image loading, and return a clean, fully-typed JSON structure:\n\n```\n[\n  {\n    \"title\": \"Why I still use Vim\",\n    \"url\": \"https://example.com/vim\",\n    \"points\": 142\n  },\n  {\n    \"title\": \"Show HN: Opticparse - AI Visual Scraper\",\n    \"url\": \"https://github.com/parastejpal987-cmyk/opticparse\",\n    \"points\": 98\n  }\n]\n```\n\nThe SDK client supports configuring the browser environment to handle dynamic loading states:\n\n``` js\ntypescript\nconst result = await client.scrape({\n  targetUrl: 'https://example.com',\n  extractionQuery: 'Extract details...',\n\n  // Custom screen sizes for responsive layouts\n  viewportWidth: 1920,\n  viewportHeight: 1080,\n\n  // Wait until page is completely loaded ('networkidle' | 'load' | 'domcontentloaded')\n  waitUntil: 'networkidle',\n\n  // Adjust timeout threshold (in milliseconds) for slower connec\n```\n\n", "url": "https://wpnews.pro/news/scraping-dynamic-web-pages-without-selectors-using-ai-vision-typescript-tutorial", "canonical_source": "https://dev.to/parastejpal987cmyk/scraping-dynamic-web-pages-without-selectors-using-ai-vision-typescriptjavascript-tutorial-1n63", "published_at": "2026-06-19 05:29:11+00:00", "updated_at": "2026-06-19 05:30:11.663534+00:00", "lang": "en", "topics": ["artificial-intelligence", "computer-vision", "developer-tools", "ai-tools", "natural-language-processing"], "entities": ["Opticparse", "Gemini", "Hacker News", "RapidAPI"], "alternates": {"html": "https://wpnews.pro/news/scraping-dynamic-web-pages-without-selectors-using-ai-vision-typescript-tutorial", "markdown": "https://wpnews.pro/news/scraping-dynamic-web-pages-without-selectors-using-ai-vision-typescript-tutorial.md", "text": "https://wpnews.pro/news/scraping-dynamic-web-pages-without-selectors-using-ai-vision-typescript-tutorial.txt", "jsonld": "https://wpnews.pro/news/scraping-dynamic-web-pages-without-selectors-using-ai-vision-typescript-tutorial.jsonld"}}