# Building a Music Brain: My YouTube Data Workflow

> Source: <https://promptcube3.com/en/threads/3717/>
> Published: 2026-07-26 14:46:48+00:00

# Building a Music Brain: My YouTube Data Workflow

This isn't a quick weekend build. It's a multi-month deployment involving a browser extension for data collection, a backend for storage, and ML pipelines for analysis. The core goal is to capture "implicit signals"—the things I do without thinking, like skipping a track after four seconds or looping a specific chorus—and translate those into a behavioral profile.

## The Data Pipeline Architecture

To make this work, the system has to track the lifecycle of a single song from the moment I click it to the moment it influences a playlist.

1. **Data Acquisition (Browser Extension)**

A content script monitors the YouTube video element. Instead of just tracking "views," it acts as a stenographer, recording `timeupdate`

, `pause`

, `seeked`

, and `ended`

events. This allows the system to calculate the actual watch percentage and identify specific segments I find appealing.

2. **Ingestion (Event API)**

To avoid overloading the server, the extension batches these observations into JSON payloads and POSTs them to a FastAPI backend:

```
{
  "video_id": "xyz123",
  "title": "Artist - Song Name",
  "channel": "OfficialChannel",
  "watch_segments": [[0, 41], [132, 222]],
  "timestamp": "2023-10-27T21:47:00Z",
  "session_id": "sess_98765"
}
```

3. **Storage (PostgreSQL)**

The raw events are written directly to PostgreSQL. I keep the raw signal intact because any future changes to the ML logic will require re-processing the original data.

4. **Processing (Offline ML Pipeline)**

This is where the "forensic" work happens. A nightly pipeline processes the raw events to:

**Classify:** Confirm if the video is actually music.**Resolve:** Parse titles and use external lookups for clean metadata.**Analyze:** Use audio embeddings and mood tagging to determine the "vibe."**Score:** Convert watch percentages and replay counts into implicit ratings.

5.

**Indexing (Vector Store)**

The final step involves pushing audio embeddings into

`pgvector`

. By storing these as vectors alongside relational metadata, the system can perform similarity searches to find songs that match the specific "fingerprint" of my late-night synthwave sessions or morning lo-fi habits.This setup transforms a messy history page into a structured AI workflow, allowing for a level of personalization that generic algorithms usually miss.

[Next Swarmvault: A Lean LLM Agent Workflow →](/en/threads/3702/)