{"slug": "translating-300-page-books-with-claude-taming-token-limits-and-context-windows", "title": "Translating 300-Page Books with Claude: Taming Token Limits and Context Windows", "summary": "LectuLibre, an AI-powered book translation platform, developed a chunking and orchestration system to translate 300-page books using Claude API while managing token limits. The system uses paragraph- and sentence-aware splitting with overlap and a glossary of named entities to maintain translation quality and narrative consistency across chunks.", "body_md": "*How we chunk long-form content and maintain translation quality with Claude API*\n\nWhen we launched LectuLibre, our AI-powered book translation platform, we thought the hard part would be fine-tuning translation quality. It turned out the real engineering challenge was more mundane: Claude's token limits.\n\nA 300-page novel contains roughly 120,000 tokens. Claude 3 Sonnet has a 200K context window, so you might assume you can just send the whole book and ask for a translation. But the output token limit is only 4,096 tokens (8,192 for some models). Even if the input fits, asking for a 120,000-token translation in one call will hit a wall. The API will return a truncated response or an error.\n\nIn this post, I'll walk through the chunking and orchestration system we built to translate long-form content with Claude reliably and cost-effectively, without losing narrative consistency.\n\nBooks are naturally long. A typical 300-page EPUB produces around 100,000–150,000 tokens. Claude's context window can hold that input, but the model's maximum output tokens are capped (4,096 for Sonnet). You cannot ask the model to output a full book translation in one API call. Even if you could, quality suffers: the model may lose focus, repeat content, or hallucinate details over very long generations.\n\nWe needed a system that would:\n\nOur first naive attempt used `textwrap` or simple character-based slicing. That produced chunks ending mid-sentence, which led to awkward translations and lost pronouns. We quickly moved to a paragraph- and sentence-aware splitter.\n\nWe use `tiktoken` with the `cl100k_base` encoding as a fast approximation for Claude's token count. It's not exact, but close enough for chunk sizing. The actual `anthropic` SDK also provides a `count_tokens` method if you need precision.\n\nHere's the chunker we use:\n\n``` python\nimport tiktoken\nimport re\n\nenc = tiktoken.get_encoding('cl100k_base')\n\ndef count_tokens(text: str) -> int:\n    return len(enc.encode(text))\n\ndef split_into_chunks(text: str, max_tokens=8000, overlap_tokens=200) -> list[str]:\n    paragraphs = re.split(r'\\n\\s*\\n', text)\n    chunks = []\n    current = ''\n    current_tokens = 0\n    for para in paragraphs:\n        para_tokens = count_tokens(para)\n        if current_tokens + para_tokens > max_tokens and current:\n            chunks.append(current)\n            # keep last overlap_tokens worth from current as overlap\n            overlap_text = current[-overlap_tokens:] if len(current) > overlap_tokens else current\n            current = overlap_text\n            current_tokens = count_tokens(current)\n        current += '\\n\\n' + para\n        current_tokens = count_tokens(current)\n    if current:\n        chunks.append(current)\n    return chunks\n```\n\nWe set `max_tokens=8000` for source chunks. That leaves room for the system prompt and any glossary we inject, while keeping the expected translation output under 4,096 tokens. The overlap of 200 tokens (about 150 English words) ensures the next chunk starts with a bit of context already seen, reducing boundary errors.\n\nA subtle improvement: we also split paragraphs into sentences using `nltk.sent_tokenize` before building chunks if a single paragraph is longer than the max token limit. That way we never have a chunk that exceeds the limit because of one giant paragraph.\n\nTranslation quality depends heavily on continuity. Names, places, and invented terms must stay consistent. Our approach is to include a running context in the prompt for each chunk. This context consists of:\n\nWe build the glossary once before translation using spaCy:\n\n``` python\nimport spacy\nnlp = spacy.load('en_core_web_sm')\n\ndef build_glossary(text: str, max_terms=100) -> str:\n    doc = nlp(text[:100000])  # limit for speed\n    names = set()\n    for ent in doc.ents:\n        if ent.label_ in ['PERSON', 'ORG', 'GPE', 'LOC']:\n            names.add(ent.text)\n    return ', '.join(sorted(names)[:max_terms])\n```\n\nThen, for each chunk translation, we prepend the glossary and previous context to the prompt:\n\n```\nprompt = f'''Translate the following book text from English to {target_lang}.\nMaintain style, tone, and terminology. Use the glossary:\n{glossary_str}\n\nPrevious translated context (for consistency):\n{prev_context}\n\nSource text:\n{chunk}\n'''\n```\n\nThis two-part context (glossary + previous tail) dramatically reduced name inconsistencies in our tests. Initially we saw character names translated differently in later chapters; after adding the glossary, those errors dropped to near zero.\n\nClaude's API has rate limits (requests per minute and tokens per minute). To speed up translation while respecting those limits, we use `anthropic.AsyncAnthropic` with an `asyncio.Semaphore`. We typically run 4 concurrent requests for Claude Sonnet.\n\nHere's the core translation function:\n\n``` python\nimport asyncio\nimport anthropic\n\nclient = anthropic.AsyncAnthropic(api_key='...')\n\nasync def translate_chunk(chunk: str, prev_context: str, glossary_str: str, target_lang: str, semaphore: asyncio.Semaphore) -> str:\n    async with semaphore:\n        prompt = f'''Translate the following book text from English to {target_lang}.\nMaintain style, tone, and terminology. Use the glossary:\n{glossary_str}\n\nPrevious translated context (for consistency):\n{prev_context}\n\nSource text:\n{chunk}\n'''\n        for attempt in range(3):\n            try:\n                response = await client.messages.create(\n                    model='claude-3-sonnet-20240229',\n                    max_tokens=4096,\n                    temperature=0.3,\n                    system='You are a professional literary translator.',\n                    messages=[{'role': 'user', 'content': prompt}]\n                )\n                return response.content[0].text\n            except anthropic.RateLimitError:\n                await asyncio.sleep(60 * (attempt + 1))\n        raise RuntimeError('Rate limit retries exhausted')\n```\n\nThe retry loop handles transient 429 errors. We also catch `anthropic.APIStatusError` for server-side overloads.\n\nThe main loop processes chunks sequentially to maintain context, but the actual API calls are concurrent within each chunk? Actually, we process chunks one by one in order because each chunk's context depends on the previous translation. To get some parallelism, we can translate chunks in batches where each batch shares the same previous context, but we found that reduced quality slightly. For now, sequential chunk translation with a concurrency of 1 per book is simpler and only takes about 2–3 minutes for a 300-page book using Sonnet.\n\nIf you need higher throughput, you could translate independent sections in parallel (e.g., different chapters) and then merge, but you lose the running context.\n\nFor a 300-page novel (~120,000 tokens), our chunker produces around 16–20 chunks of 8,000 tokens with overlap. Translating to Spanish with Claude 3 Sonnet:\n\nWe experimented with Claude 3 Haiku for a cheaper option ($0.25/M input, $1.25/M output). It cost about $0.35 per book but produced noticeably worse translations, especially for literary text with figurative language. For technical or non-fiction books, Haiku is acceptable; for novels, we stick with Sonnet.\n\nPDF files are the wild west. Headers, footers, page numbers, and two-column layouts wreak havoc on chunking. We use PyMuPDF (`fitz`) because it gives us block-level text with bounding boxes, allowing us to filter out headers/footers based on vertical position. For EPUB, we use `ebooklib` to iterate over spine items and extract HTML, then strip tags with BeautifulSoup. This preserves chapter boundaries, which we use as natural chunk boundaries before applying token-based splitting.\n\nOne hard-won lesson: always remove page numbers and repeated header text before tokenizing the full text. Otherwise, those artifacts become part of the chunks and get translated, producing garbage in the middle of chapters.\n\nWhat worked:\n\nWhat didn't:\n\nOpen question for the community: Have you found a reliable way to translate poetry, code, or tables inside long documents without breaking the surrounding narrative? We currently treat them as opaque blocks and translate them separately, but integration remains rough.\n\nBuilding LectuLibre taught us that long-form LLM translation is less about model capability and more about solid engineering around context management. If you're building something similar, start with robust chunking and context propagation—the model will handle the rest.", "url": "https://wpnews.pro/news/translating-300-page-books-with-claude-taming-token-limits-and-context-windows", "canonical_source": "https://dev.to/jacob_gong/translating-300-page-books-with-claude-taming-token-limits-and-context-windows-dj5", "published_at": "2026-09-09 03:01:49+00:00", "updated_at": "2026-09-09 03:19:48.920190+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "ai-products"], "entities": ["LectuLibre", "Claude", "Anthropic", "tiktoken", "spaCy", "nltk"], "alternates": {"html": "https://wpnews.pro/news/translating-300-page-books-with-claude-taming-token-limits-and-context-windows", "markdown": "https://wpnews.pro/news/translating-300-page-books-with-claude-taming-token-limits-and-context-windows.md", "text": "https://wpnews.pro/news/translating-300-page-books-with-claude-taming-token-limits-and-context-windows.txt", "jsonld": "https://wpnews.pro/news/translating-300-page-books-with-claude-taming-token-limits-and-context-windows.jsonld"}}