{"slug": "crawaler-finishing-my-ai-powered-price-comparison-platform", "title": "Crawaler: Finishing My AI-Powered Price Comparison Platform", "summary": "A developer has launched Crawaler, an open-source, AI-powered price comparison platform for Pakistani e-commerce. The tool searches products across Daraz and Telemart, groups identical items, and displays prices side-by-side, while also supporting AI image search via Google Gemini. The platform, built with React, Node.js, and MongoDB, solves the problem of manually checking multiple websites for the best deals.", "body_md": "**What I Built**\n\n**Crawaler** is an open-source, AI-powered price comparison platform for Pakistani e-commerce.\n\nIt searches products across **Daraz** and **Telemart**, groups identical products together, and displays prices side by side so users can instantly find the best deal without opening multiple tabs.\n\nThe platform also supports **AI image search**. Users can upload a product photo, and Google Gemini identifies the product and automatically performs the search across supported stores.\n\n🔗 **Live Link:** [https://crawaler-front-v2.vercel.app](https://crawaler-front-v2.vercel.app)\n\n| Layer | Technologies |\n|---|---|\n| Frontend | React, Vite, TypeScript, Tailwind CSS, shadcn/ui |\n| Backend | Node.js, Express, Puppeteer, Cheerio |\n| Database | MongoDB, Mongoose |\n| AI | Google Gemini |\n| Authentication | JWT, bcrypt, Nodemailer |\n\nOnline buyers often spend a lot of time checking different websites to compare prices. Crawaler solves this problem by bringing everything into a single platform.\n\nThis project started as a side project that remained unfinished for months.\n\nWhen I returned to it, the Daraz integration was already working, but a comparison platform needs multiple sources to provide real value. Finding and integrating a reliable second platform became the biggest challenge.\n\nI first experimented with OLX and managed to extract some data, but maintaining a stable integration proved difficult. Eventually, I decided to move on.\n\nNext, I explored Temu. Their APIs and requests were heavily protected and obfuscated, making reliable integration extremely challenging. After several attempts, I abandoned this approach.\n\nFinally, I turned to Telemart. While it exposed API, access tokens expired frequently, causing the scraper to fail during execution.\n\nTo solve this, I implemented automatic token refresh handling, allowing Crawaler to detect expired tokens and recover seamlessly. This became the reliable second data source the platform needed.\n\nWith comparison functionality complete, I added the feature I had wanted from the beginning: AI-powered image search.\n\nBuyers can upload a product image, Google Gemini identifies the product, and Crawaler automatically searches and compares prices across supported platforms.\n\nGitHub Copilot played an important role in helping me finish the project.\n\nIt assisted with debugging scraper issues, improving reliability, generating boilerplate code, refining React components, and accelerating development when I was stuck on implementation details.\n\nCopilot helped me focus more on solving complex engineering challenges while spending less time on repetitive tasks. It was especially useful while refining scraping logic, handling token refresh workflows, and improving overall code quality.\n\nCrawaler transformed from an unfinished side project into a fully functional platform that helps buyers save both time and money.\n\nThe GitHub Finish-Up-A-Thon Challenge provided the motivation I needed to finally complete something that had been sitting unfinished for months. I'm excited to continue improving the platform, add more store integrations, and welcome contributions from the open-source community.", "url": "https://wpnews.pro/news/crawaler-finishing-my-ai-powered-price-comparison-platform", "canonical_source": "https://dev.to/rehmanstackdev/crawaler-finishing-my-ai-powered-price-comparison-platform-2mbj", "published_at": "2026-06-04 06:04:48+00:00", "updated_at": "2026-06-04 06:11:44.446491+00:00", "lang": "en", "topics": ["ai-products", "ai-tools", "ai-startups"], "entities": ["Crawaler", "Daraz", "Telemart", "Google Gemini", "OLX", "Temu", "MongoDB", "Node.js"], "alternates": {"html": "https://wpnews.pro/news/crawaler-finishing-my-ai-powered-price-comparison-platform", "markdown": "https://wpnews.pro/news/crawaler-finishing-my-ai-powered-price-comparison-platform.md", "text": "https://wpnews.pro/news/crawaler-finishing-my-ai-powered-price-comparison-platform.txt", "jsonld": "https://wpnews.pro/news/crawaler-finishing-my-ai-powered-price-comparison-platform.jsonld"}}