Extract Plain Text from Medium Posts for RAG and Search Indexes

A developer created an API that extracts clean plain text from Medium articles, stripping navigation, clap bars, and scripts for use in embeddings, summarization, and full-text search. The tool provides separate endpoints for article content and metadata, enabling chunking of text for vector databases and RAG pipelines. The solution supports integration with OpenAI embeddings, Ollama, or other models for LLM training and retrieval applications.

Chunk clean article content for embeddings, summarization, and full-text search—skip nav, clap bars, and scripts. HTML embeds are for humans; plain text is for chunking, embeddings, and summarization. One call should return body text without nav, clap bars, or script tags. Tool outcome: ingest-medium-article.ts → chunked documents in your vector DB. GET /article/{id}/content → plain text. GET /article/{id} for title, tags, author metadata. js const API = 'https://api.zenndra.com'; const headers = { Authorization: Bearer ${process.env.ZENNDRA API KEY} }; export async function fetchArticleText articleId { const contentRes, metaRes = await Promise.all fetch ${API}/article/${articleId}/content , { headers } , fetch ${API}/article/${articleId} , { headers } , ; const { content } = await contentRes.json ; const meta = await metaRes.json ; return { id: articleId, title: meta.title, tags: meta.tags, text: content, }; } export function chunkText text, { size = 800, overlap = 100 } = {} { const words = text.split /\s+/ ; const chunks = ; for let i = 0; i < words.length; i += size - overlap { chunks.push words.slice i, i + size .join ' ' ; } return chunks.filter Boolean ; } Wire chunkText to OpenAI embeddings https://platform.openai.com/docs/guides/embeddings , Ollama https://ollama.com/ , or your host’s model—swap the vector client, keep the ingest shape. article id and chunk index in metadata for citations.For human-readable syndication, see embed articles https://./embed-medium-articles-on-website.md —different threat model than LLM training. medium plain text api , medium rag pipeline , medium embeddings , medium article content extraction , llm medium .