{"slug": "building-a-music-brain-my-youtube-data-workflow", "title": "Building a Music Brain: My YouTube Data Workflow", "summary": "A developer is building a multi-month YouTube data workflow that uses a browser extension to capture implicit listening signals—such as skipping a track after four seconds or looping a chorus—and processes them through a FastAPI backend, PostgreSQL storage, and an offline ML pipeline to generate personalized playlists. The system classifies videos, resolves metadata, analyzes audio embeddings with mood tagging, and stores vectors in pgvector for similarity searches, aiming to achieve a level of personalization beyond generic algorithms.", "body_md": "# Building a Music Brain: My YouTube Data Workflow\n\nThis isn't a quick weekend build. It's a multi-month deployment involving a browser extension for data collection, a backend for storage, and ML pipelines for analysis. The core goal is to capture \"implicit signals\"—the things I do without thinking, like skipping a track after four seconds or looping a specific chorus—and translate those into a behavioral profile.\n\n## The Data Pipeline Architecture\n\nTo make this work, the system has to track the lifecycle of a single song from the moment I click it to the moment it influences a playlist.\n\n1. **Data Acquisition (Browser Extension)**\n\nA content script monitors the YouTube video element. Instead of just tracking \"views,\" it acts as a stenographer, recording `timeupdate`\n\n, `pause`\n\n, `seeked`\n\n, and `ended`\n\nevents. This allows the system to calculate the actual watch percentage and identify specific segments I find appealing.\n\n2. **Ingestion (Event API)**\n\nTo avoid overloading the server, the extension batches these observations into JSON payloads and POSTs them to a FastAPI backend:\n\n```\n{\n  \"video_id\": \"xyz123\",\n  \"title\": \"Artist - Song Name\",\n  \"channel\": \"OfficialChannel\",\n  \"watch_segments\": [[0, 41], [132, 222]],\n  \"timestamp\": \"2023-10-27T21:47:00Z\",\n  \"session_id\": \"sess_98765\"\n}\n```\n\n3. **Storage (PostgreSQL)**\n\nThe raw events are written directly to PostgreSQL. I keep the raw signal intact because any future changes to the ML logic will require re-processing the original data.\n\n4. **Processing (Offline ML Pipeline)**\n\nThis is where the \"forensic\" work happens. A nightly pipeline processes the raw events to:\n\n**Classify:** Confirm if the video is actually music.**Resolve:** Parse titles and use external lookups for clean metadata.**Analyze:** Use audio embeddings and mood tagging to determine the \"vibe.\"**Score:** Convert watch percentages and replay counts into implicit ratings.\n\n5.\n\n**Indexing (Vector Store)**\n\nThe final step involves pushing audio embeddings into\n\n`pgvector`\n\n. By storing these as vectors alongside relational metadata, the system can perform similarity searches to find songs that match the specific \"fingerprint\" of my late-night synthwave sessions or morning lo-fi habits.This setup transforms a messy history page into a structured AI workflow, allowing for a level of personalization that generic algorithms usually miss.\n\n[Next Swarmvault: A Lean LLM Agent Workflow →](/en/threads/3702/)", "url": "https://wpnews.pro/news/building-a-music-brain-my-youtube-data-workflow", "canonical_source": "https://promptcube3.com/en/threads/3717/", "published_at": "2026-07-26 14:46:48+00:00", "updated_at": "2026-07-26 15:11:56.509432+00:00", "lang": "en", "topics": ["machine-learning", "ai-infrastructure", "ai-tools"], "entities": ["YouTube", "FastAPI", "PostgreSQL", "pgvector"], "alternates": {"html": "https://wpnews.pro/news/building-a-music-brain-my-youtube-data-workflow", "markdown": "https://wpnews.pro/news/building-a-music-brain-my-youtube-data-workflow.md", "text": "https://wpnews.pro/news/building-a-music-brain-my-youtube-data-workflow.txt", "jsonld": "https://wpnews.pro/news/building-a-music-brain-my-youtube-data-workflow.jsonld"}}