{"slug": "take-the-well-maintained-road-over-risky-shortcuts-ditching-web-scraping-for-an", "title": "\"Take the Well-Maintained Road Over Risky Shortcuts\": Ditching Web Scraping for an AI App Powered by Official RSS & Gemini (Lessons from Logistics)", "summary": "A Japanese truck driver self-studying Python built 'Puoppo,' an AI trend analysis app that uses official RSS feeds instead of web scraping to avoid IP blocks and maintenance costs. The system, powered by Google's Gemini 2.5 Flash model, aggregates up to 100 articles per query and presents them in a single scrollable pane for efficient morning news consumption.", "body_md": "**GitHub Repository:** [tosane932 / puoppo_app](https://github.com/tosane932/puoppo_app)\n\nAn active truck driver in Japan self-studying Python, leveraging logistics domain knowledge to transition into a Web Engineer.\n\nIn this article, I share the design philosophy and technical choices behind \"Puoppo,\" an automated AI trend analysis system I built during my 42nd hour of studying Python (with no IT background or prior development experience).\n\nRather than a gimmick chasing short-lived trends, the system is built on a timeless design philosophy—like a reliable pair of jeans that never wears out—focused on \"reliably controlling domain data from the field.\"\n\nIn daily technical trend and news gathering, visiting multiple sources and digesting noisy, long articles became a major bottleneck. I needed a mechanism to consume high-density primary information within a limited timeframe each morning (e.g., the few minutes it takes to cook a Japanese omelet).\n\nSimply enter topics you want to know about (e.g., politics, inflation, latest music news) into a text box, and the AI actively collects and structures up to 100 related articles. You can digest everything in a single screen pane just by scrolling.\n\n`gemini-2.5-flash`\n\n)The actual running environment and UI design are available in the demo video below (YouTube: 7m 49s).\n\n*(Click the image to watch on YouTube)*\n\nInitial designs tested direct web scraping from major news sites, but IP blocking and maintenance costs due to layout changes presented major hurdles.\n\nInstead of trying to technically \"bypass\" blocks, I pivoted to leveraging official RSS feeds provided by publisher media.\n\nThis decision stems from practical logistics experience: **\"Taking a well-maintained, reliable road yields better overall efficiency (speed) than forcing a shortcut.\"** This allowed the immediate setup of a safe and dependable data pipeline.\n\nThe core value of this system is instantaneous data flow: aggregating data from dynamic keywords, passing it to an LLM, and returning results.\n\nSince authentication and ORM were unnecessary for this phase, I chose Flask, a micro-framework that enables rapid routing setup. By stripping away over-engineering, I rolled out the prototype in just one day (~10 hours).\n\nInjecting context from up to 100 article texts in a single request required an LLM model with a vast token capacity, making Gemini an obvious choice.\n\nFurthermore, balancing rapid response times (10–15 seconds) with excellent cost-effectiveness for personal projects led to selecting `gemini-2.5-flash`\n\n, a lightweight model selected as of May 2026.\n\nRather than building component architectures using modern JavaScript frameworks (React/Vue), I implemented it with plain HTML/CSS.\n\nBy eliminating resource expenditure on complex UI layouts, I prioritized pure utility: \"allowing users to absorb accurate information smoothly into their minds without stopping their scroll.\"\n\nSteps to run locally in a virtual environment (Linux / macOS):\n\n```\n# 1. Clone the repository\ngit clone [https://github.com/tosane932/puoppo_app.git](https://github.com/tosane932/puoppo_app.git)\ncd puoppo_app\n\n# 2. Create and activate virtual environment\npython3 -m venv .venv\nsource .venv/bin/activate\n\n# 3. Install required packages\npip install -r requirements.txt\n\n# 4. Set API Key\nexport GEMINI_API_KEY=\"YOUR_API_KEY\"\n\n# 5. Run the app\npython app.py\n```\n\nSteps to run locally on Windows:\n\n```\n# 1. Clone the repository\ngit clone [https://github.com/tosane932/puoppo_app.git](https://github.com/tosane932/puoppo_app.git)\ncd puoppo_app\n\n# 2. Create and activate virtual environment\npython -m venv .venv\n# [For PowerShell]\n.venv\\Scripts\\Activate.ps1\n# [For Command Prompt]\n.venv\\Scripts\\activate.bat\n\n# 3. Install required packages\npip install -r requirements.txt\n\n# 4. Set API Key\n# [For PowerShell]\n$env:GEMINI_API_KEY=\"YOUR_API_KEY\"\n# [For Command Prompt]\nset GEMINI_API_KEY=YOUR_API_KEY\n\n# 5. Run the app\npython app.py\n```\n\nWhile current capabilities satisfy initial requirements and deliver reliable results, I plan a two-stage refactoring process by the end of 2026 to elevate the system into production-ready quality:\n\nI will organize and decouple the logic currently centralized in Flask (RSS parsing, LLM control, view control), transitioning toward a decoupled layered architecture inspired by Clean Architecture.\n\nTo adapt to changes in external APIs (Gemini) or RSS feed formats, I will introduce unit tests and mock tests using Pytest, creating a robust codebase oriented around Continuous Integration (CI).\n\nThrough the development of \"Puoppo,\" I reconfirmed the power of minimalism: \"simply combining only the necessary components to solve the specific problem in front of you,\" without getting swayed by over-engineering or trending tech stacks.\n\nReducing wasted time spent searching for news, and using created resources to decide what to build next—this system reminds me of the true joy of engineering.\n\n**GitHub Repository:** [tosane932 / puoppo_app](https://github.com/tosane932/puoppo_app)", "url": "https://wpnews.pro/news/take-the-well-maintained-road-over-risky-shortcuts-ditching-web-scraping-for-an", "canonical_source": "https://dev.to/tosane932/take-the-well-maintained-road-over-risky-shortcuts-ditching-web-scraping-for-an-ai-app-powered-3gn4", "published_at": "2026-07-25 20:11:43+00:00", "updated_at": "2026-07-25 20:31:43.759483+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "developer-tools"], "entities": ["Google", "Gemini", "Flask", "Puoppo"], "alternates": {"html": "https://wpnews.pro/news/take-the-well-maintained-road-over-risky-shortcuts-ditching-web-scraping-for-an", "markdown": "https://wpnews.pro/news/take-the-well-maintained-road-over-risky-shortcuts-ditching-web-scraping-for-an.md", "text": "https://wpnews.pro/news/take-the-well-maintained-road-over-risky-shortcuts-ditching-web-scraping-for-an.txt", "jsonld": "https://wpnews.pro/news/take-the-well-maintained-road-over-risky-shortcuts-ditching-web-scraping-for-an.jsonld"}}