{"slug": "build-an-ai-mentor-from-your-favorite-youtube-creator-pull-every-transcript-as", "title": "Build an AI mentor from your favorite YouTube creator: pull every transcript as timestamped Markdown for RAG (Brian Jenney / Parsity)", "summary": "Brian Jenney, creator of the Develop Yourself podcast, published a Python script that pulls every transcript from a YouTube channel, playlist, or list of videos and writes them as timestamped Markdown files for retrieval-augmented generation. The tool groups caption lines into roughly 60-second passages so RAG answers can cite the exact episode and moment, and emits a manifest of what succeeded or was skipped. The output folder is designed to be dropped into Hippocampus or any other RAG tool.", "body_md": "|  | #!/usr/bin/env python3 | \n|  | \"\"\" | \n|  | Build an AI mentor from your favorite creator. | \n|  |  | \n|  | Pulls every transcript from a YouTube channel, playlist, or list of videos and | \n|  | writes them as clean, timestamped Markdown files. Drop the output folder into | \n|  | Hippocampus (or any RAG tool) and ask it questions. Every answer can cite the | \n|  | episode and the exact moment it came from. | \n|  |  | \n|  | By Brian Jenney (youtube.com/@brianjenney), from the video where he turned | \n|  | every Develop Yourself podcast interview into an AI career mentor. | \n|  |  | \n|  | Learn to build RAG systems like this yourself: https://parsity.io | \n|  | Try Hippocampus (20% off for 12 months, code BrianJenney): | \n|  | https://www.thalius.ai/?utm_source=youtube&utm_medium=influeri&utm_campaign=thalius_hippocampus_developer_pilot&utm_content=brianjenney | \n|  |  | \n|  | Setup (a virtual environment avoids \"externally-managed-environment\" errors on Mac): | \n|  | python3 -m venv .venv && source .venv/bin/activate | \n|  | pip install yt-dlp youtube-transcript-api | \n|  |  | \n|  | Usage: | \n|  | python build_mentor_corpus.py https://www.youtube.com/@brianjenney | \n|  | python build_mentor_corpus.py https://www.youtube.com/@brianjenney --limit 50 | \n|  | python build_mentor_corpus.py \"https://www.youtube.com/playlist?list=PLxxxx\" --out my_mentor | \n|  | python build_mentor_corpus.py https://youtu.be/VIDEO_ID https://youtu.be/OTHER_ID | \n|  |  | \n|  | Output: | \n|  | mentor_corpus/ | \n|  | 2026-10-28 - Episode title - VIDEO_ID.md   (one file per video) | \n|  | ... | \n|  | _manifest.json                              (what worked, what got skipped) | \n|  | \"\"\" | \n|  |  | \n|  | import argparse | \n|  | import json | \n|  | import re | \n|  | import sys | \n|  | import time | \n|  | from pathlib import Path | \n|  |  | \n|  | try: | \n|  | from yt_dlp import YoutubeDL | \n|  | from youtube_transcript_api import YouTubeTranscriptApi | \n|  | except ImportError: | \n|  | sys.exit( | \n|  | \"Missing dependencies. Run:\\n\" | \n|  | \"  python3 -m venv .venv && source .venv/bin/activate\\n\" | \n|  | \"  pip install yt-dlp youtube-transcript-api\" | \n|  | ) | \n|  |  | \n|  |  | \n|  | # Group caption lines into ~60-second passages. Long enough to carry a full | \n|  | # thought, short enough that a citation points somewhere useful. | \n|  | PASSAGE_SECONDS = 60 | \n|  |  | \n|  |  | \n|  | def list_videos(urls, limit): | \n|  | \"\"\"Expand channels/playlists/single videos into a flat list of video metadata.\"\"\" | \n|  | opts = {\"quiet\": True, \"extract_flat\": \"in_playlist\", \"skip_download\": True} | \n|  | videos = [] | \n|  | with YoutubeDL(opts) as ydl: | \n|  | for url in urls: | \n|  | # A channel URL lists tabs (videos, shorts, live). Ask for long-form videos. | \n|  | if re.search(r\"youtube\\.com/(@\\|c/\\|channel/\\|user/)\", url) and not url.rstrip(\"/\").endswith( | \n|  | (\"/videos\", \"/streams\", \"/shorts\") | \n|  | ): | \n|  | url = url.rstrip(\"/\") + \"/videos\" | \n|  | info = ydl.extract_info(url, download=False) | \n|  | entries = info.get(\"entries\") or [info] | \n|  | for e in entries: | \n|  | if not e or not e.get(\"id\"): | \n|  | continue | \n|  | videos.append({\"id\": e[\"id\"], \"title\": e.get(\"title\") or e[\"id\"]}) | \n|  | if limit and len(videos) >= limit: | \n|  | return videos | \n|  | return videos | \n|  |  | \n|  |  | \n|  | def video_details(video_id): | \n|  | \"\"\"Grab title, channel and upload date for one video (cheap, no download).\"\"\" | \n|  | opts = {\"quiet\": True, \"skip_download\": True, \"ignore_no_formats_error\": True, \"no_warnings\": True} | \n|  | with YoutubeDL(opts) as ydl: | \n|  | info = ydl.extract_info(f\"https://www.youtube.com/watch?v={video_id}\", download=False) | \n|  | date = info.get(\"upload_date\") or \"\" | \n|  | if len(date) == 8: | \n|  | date = f\"{date[:4]}-{date[4:6]}-{date[6:]}\" | \n|  | return { | \n|  | \"title\": info.get(\"title\") or video_id, | \n|  | \"channel\": info.get(\"channel\") or info.get(\"uploader\") or \"\", | \n|  | \"date\": date, | \n|  | \"description\": (info.get(\"description\") or \"\").strip(), | \n|  | } | \n|  |  | \n|  |  | \n|  | def fetch_transcript(video_id, languages): | \n|  | \"\"\"Return a list of {start, text} caption lines, or None if the video has no captions.\"\"\" | \n|  | api = YouTubeTranscriptApi() | \n|  | try: | \n|  | fetched = api.fetch(video_id, languages=languages) | \n|  | except Exception: | \n|  | # Fall back to whatever transcript exists (auto-generated, other language). | \n|  | try: | \n|  | transcript = next(iter(api.list(video_id))) | \n|  | fetched = transcript.fetch() | \n|  | except Exception: | \n|  | return None | \n|  | return [{\"start\": s.start, \"text\": s.text} for s in fetched] | \n|  |  | \n|  |  | \n|  | def clock(seconds): | \n|  | seconds = int(seconds) | \n|  | h, rem = divmod(seconds, 3600) | \n|  | m, s = divmod(rem, 60) | \n|  | return f\"{h}:{m:02d}:{s:02d}\" if h else f\"{m}:{s:02d}\" | \n|  |  | \n|  |  | \n|  | def to_passages(lines): | \n|  | \"\"\"Merge caption fragments into ~60-second passages with clean text.\"\"\" | \n|  | passages, current, start = [], [], None | \n|  | for line in lines: | \n|  | text = re.sub(r\"\\[(music\\|applause\\|laughter)\\]\", \"\", line[\"text\"], flags=re.I) | \n|  | text = re.sub(r\"\\s+\", \" \", text.replace(\"\\n\", \" \")).strip() | \n|  | if not text: | \n|  | continue | \n|  | if start is None: | \n|  | start = line[\"start\"] | \n|  | current.append(text) | \n|  | if line[\"start\"] - start >= PASSAGE_SECONDS: | \n|  | passages.append((start, \" \".join(current))) | \n|  | current, start = [], None | \n|  | if current: | \n|  | passages.append((start, \" \".join(current))) | \n|  | return passages | \n|  |  | \n|  |  | \n|  | def to_markdown(video_id, meta, passages): | \n|  | url = f\"https://www.youtube.com/watch?v={video_id}\" | \n|  | out = [ | \n|  | f\"# {meta['title']}\", | \n|  | \"\", | \n|  | f\"- Source: {url}\", | \n|  | f\"- Creator: {meta['channel']}\" if meta[\"channel\"] else None, | \n|  | f\"- Published: {meta['date']}\" if meta[\"date\"] else None, | \n|  | \"\", | \n|  | ] | \n|  | if meta[\"description\"]: | \n|  | # The first lines of a description usually name the guest and topic, | \n|  | # which helps retrieval a lot. Skip the link dump below it. | \n|  | summary = meta[\"description\"].split(\"\\n\\n\")[0][:600] | \n|  | out += [\"## About this episode\", \"\", summary, \"\"] | \n|  | out += [\"## Transcript\", \"\"] | \n|  | for start, text in passages: | \n|  | stamp = clock(start) | \n|  | out += [f\"### [{stamp}]({url}&t={int(start)}s)\", \"\", text, \"\"] | \n|  | return \"\\n\".join(line for line in out if line is not None) | \n|  |  | \n|  |  | \n|  | def safe_filename(name): | \n|  | name = re.sub(r'[<>:\"/\\\\\\|?*\\x00-\\x1f]', \"\", name) | \n|  | return re.sub(r\"\\s+\", \" \", name).strip()[:90] | \n|  |  | \n|  |  | \n|  | def main(): | \n|  | parser = argparse.ArgumentParser( | \n|  | description=\"Turn a YouTube creator's videos into a timestamped corpus for RAG.\" | \n|  | ) | \n|  | parser.add_argument(\"urls\", nargs=\"+\", help=\"Channel, playlist, or video URLs\") | \n|  | parser.add_argument(\"--out\", default=\"mentor_corpus\", help=\"Output folder (default: mentor_corpus)\") | \n|  | parser.add_argument(\"--limit\", type=int, default=0, help=\"Max videos to pull (default: all)\") | \n|  | parser.add_argument(\"--lang\", default=\"en\", help=\"Preferred caption languages, comma separated (default: en)\") | \n|  | parser.add_argument(\"--delay\", type=float, default=1.0, help=\"Seconds between videos, be polite (default: 1)\") | \n|  | args = parser.parse_args() | \n|  |  | \n|  | out_dir = Path(args.out) | \n|  | out_dir.mkdir(parents=True, exist_ok=True) | \n|  | languages = [l.strip() for l in args.lang.split(\",\") if l.strip()] | \n|  |  | \n|  | print(\"Finding videos...\") | \n|  | videos = list_videos(args.urls, args.limit) | \n|  | print(f\"Found {len(videos)} videos. Pulling transcripts.\\n\") | \n|  |  | \n|  | manifest = {\"saved\": [], \"skipped\": []} | \n|  | for i, v in enumerate(videos, 1): | \n|  | vid = v[\"id\"] | \n|  | existing = list(out_dir.glob(f\"*{vid}.md\")) | \n|  | if existing: | \n|  | print(f\"[{i}/{len(videos)}] already have {v['title']}\") | \n|  | manifest[\"saved\"].append({\"id\": vid, \"file\": existing[0].name}) | \n|  | continue | \n|  |  | \n|  | lines = fetch_transcript(vid, languages) | \n|  | if not lines: | \n|  | print(f\"[{i}/{len(videos)}] no captions, skipping: {v['title']}\") | \n|  | manifest[\"skipped\"].append({\"id\": vid, \"title\": v[\"title\"], \"reason\": \"no captions\"}) | \n|  | continue | \n|  |  | \n|  | try: | \n|  | meta = video_details(vid) | \n|  | except Exception: | \n|  | meta = {\"title\": v[\"title\"], \"channel\": \"\", \"date\": \"\", \"description\": \"\"} | \n|  |  | \n|  | md = to_markdown(vid, meta, to_passages(lines)) | \n|  | prefix = f\"{meta['date']} - \" if meta[\"date\"] else \"\" | \n|  | path = out_dir / f\"{prefix}{safe_filename(meta['title'])} - {vid}.md\" | \n|  | path.write_text(md, encoding=\"utf-8\") | \n|  | manifest[\"saved\"].append({\"id\": vid, \"file\": path.name}) | \n|  | print(f\"[{i}/{len(videos)}] saved {path.name}\") | \n|  | time.sleep(args.delay) | \n|  |  | \n|  | (out_dir / \"_manifest.json\").write_text(json.dumps(manifest, indent=2), encoding=\"utf-8\") | \n|  |  | \n|  | print( | \n|  | f\"\\nDone. {len(manifest['saved'])} transcripts in ./{out_dir}/, \" | \n|  | f\"{len(manifest['skipped'])} skipped (see _manifest.json).\" | \n|  | \"\\n\\nNext: upload the folder to Hippocampus (or your own RAG pipeline) and start asking questions.\" | \n|  | \"\\nWant to learn how to build the retrieval side yourself? https://parsity.io\" | \n|  | ) | \n|  |  | \n|  |  | \n|  | if __name__ == \"__main__\": | \n|  | main() |", "url": "https://wpnews.pro/news/build-an-ai-mentor-from-your-favorite-youtube-creator-pull-every-transcript-as", "canonical_source": "https://gist.github.com/BrianJenney/4960b1bdde2b04f6b3d836eefb27dc34", "published_at": "2026-11-12 19:25:25+00:00", "updated_at": "2026-11-13 03:50:34.496332+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "generative-ai", "developer-tools"], "entities": ["Brian Jenney", "Parsity", "Hippocampus", "Thalius", "YouTube", "Develop Yourself"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/build-an-ai-mentor-from-your-favorite-youtube-creator-pull-every-transcript-as", "markdown": "https://wpnews.pro/news/build-an-ai-mentor-from-your-favorite-youtube-creator-pull-every-transcript-as.md", "text": "https://wpnews.pro/news/build-an-ai-mentor-from-your-favorite-youtube-creator-pull-every-transcript-as.txt", "jsonld": "https://wpnews.pro/news/build-an-ai-mentor-from-your-favorite-youtube-creator-pull-every-transcript-as.jsonld"}}