Translating 300-Page Books with Claude: Taming Token Limits and Context Windows

LectuLibre, an AI-powered book translation platform, developed a chunking and orchestration system to translate 300-page books using Claude API while managing token limits. The system uses paragraph- and sentence-aware splitting with overlap and a glossary of named entities to maintain translation quality and narrative consistency across chunks.

How we chunk long-form content and maintain translation quality with Claude API When we launched LectuLibre, our AI-powered book translation platform, we thought the hard part would be fine-tuning translation quality. It turned out the real engineering challenge was more mundane: Claude's token limits. A 300-page novel contains roughly 120,000 tokens. Claude 3 Sonnet has a 200K context window, so you might assume you can just send the whole book and ask for a translation. But the output token limit is only 4,096 tokens 8,192 for some models . Even if the input fits, asking for a 120,000-token translation in one call will hit a wall. The API will return a truncated response or an error. In this post, I'll walk through the chunking and orchestration system we built to translate long-form content with Claude reliably and cost-effectively, without losing narrative consistency. Books are naturally long. A typical 300-page EPUB produces around 100,000–150,000 tokens. Claude's context window can hold that input, but the model's maximum output tokens are capped 4,096 for Sonnet . You cannot ask the model to output a full book translation in one API call. Even if you could, quality suffers: the model may lose focus, repeat content, or hallucinate details over very long generations. We needed a system that would: Our first naive attempt used textwrap or simple character-based slicing. That produced chunks ending mid-sentence, which led to awkward translations and lost pronouns. We quickly moved to a paragraph- and sentence-aware splitter. We use tiktoken with the cl100k base encoding as a fast approximation for Claude's token count. It's not exact, but close enough for chunk sizing. The actual anthropic SDK also provides a count tokens method if you need precision. Here's the chunker we use: python import tiktoken import re enc = tiktoken.get encoding 'cl100k base' def count tokens text: str - int: return len enc.encode text def split into chunks text: str, max tokens=8000, overlap tokens=200 - list str : paragraphs = re.split r'\n\s \n', text chunks = current = '' current tokens = 0 for para in paragraphs: para tokens = count tokens para if current tokens + para tokens max tokens and current: chunks.append current keep last overlap tokens worth from current as overlap overlap text = current -overlap tokens: if len current overlap tokens else current current = overlap text current tokens = count tokens current current += '\n\n' + para current tokens = count tokens current if current: chunks.append current return chunks We set max tokens=8000 for source chunks. That leaves room for the system prompt and any glossary we inject, while keeping the expected translation output under 4,096 tokens. The overlap of 200 tokens about 150 English words ensures the next chunk starts with a bit of context already seen, reducing boundary errors. A subtle improvement: we also split paragraphs into sentences using nltk.sent tokenize before building chunks if a single paragraph is longer than the max token limit. That way we never have a chunk that exceeds the limit because of one giant paragraph. Translation quality depends heavily on continuity. Names, places, and invented terms must stay consistent. Our approach is to include a running context in the prompt for each chunk. This context consists of: We build the glossary once before translation using spaCy: python import spacy nlp = spacy.load 'en core web sm' def build glossary text: str, max terms=100 - str: doc = nlp text :100000 limit for speed names = set for ent in doc.ents: if ent.label in 'PERSON', 'ORG', 'GPE', 'LOC' : names.add ent.text return ', '.join sorted names :max terms Then, for each chunk translation, we prepend the glossary and previous context to the prompt: prompt = f'''Translate the following book text from English to {target lang}. Maintain style, tone, and terminology. Use the glossary: {glossary str} Previous translated context for consistency : {prev context} Source text: {chunk} ''' This two-part context glossary + previous tail dramatically reduced name inconsistencies in our tests. Initially we saw character names translated differently in later chapters; after adding the glossary, those errors dropped to near zero. Claude's API has rate limits requests per minute and tokens per minute . To speed up translation while respecting those limits, we use anthropic.AsyncAnthropic with an asyncio.Semaphore . We typically run 4 concurrent requests for Claude Sonnet. Here's the core translation function: python import asyncio import anthropic client = anthropic.AsyncAnthropic api key='...' async def translate chunk chunk: str, prev context: str, glossary str: str, target lang: str, semaphore: asyncio.Semaphore - str: async with semaphore: prompt = f'''Translate the following book text from English to {target lang}. Maintain style, tone, and terminology. Use the glossary: {glossary str} Previous translated context for consistency : {prev context} Source text: {chunk} ''' for attempt in range 3 : try: response = await client.messages.create model='claude-3-sonnet-20240229', max tokens=4096, temperature=0.3, system='You are a professional literary translator.', messages= {'role': 'user', 'content': prompt} return response.content 0 .text except anthropic.RateLimitError: await asyncio.sleep 60 attempt + 1 raise RuntimeError 'Rate limit retries exhausted' The retry loop handles transient 429 errors. We also catch anthropic.APIStatusError for server-side overloads. The main loop processes chunks sequentially to maintain context, but the actual API calls are concurrent within each chunk? Actually, we process chunks one by one in order because each chunk's context depends on the previous translation. To get some parallelism, we can translate chunks in batches where each batch shares the same previous context, but we found that reduced quality slightly. For now, sequential chunk translation with a concurrency of 1 per book is simpler and only takes about 2–3 minutes for a 300-page book using Sonnet. If you need higher throughput, you could translate independent sections in parallel e.g., different chapters and then merge, but you lose the running context. For a 300-page novel ~120,000 tokens , our chunker produces around 16–20 chunks of 8,000 tokens with overlap. Translating to Spanish with Claude 3 Sonnet: We experimented with Claude 3 Haiku for a cheaper option $0.25/M input, $1.25/M output . It cost about $0.35 per book but produced noticeably worse translations, especially for literary text with figurative language. For technical or non-fiction books, Haiku is acceptable; for novels, we stick with Sonnet. PDF files are the wild west. Headers, footers, page numbers, and two-column layouts wreak havoc on chunking. We use PyMuPDF fitz because it gives us block-level text with bounding boxes, allowing us to filter out headers/footers based on vertical position. For EPUB, we use ebooklib to iterate over spine items and extract HTML, then strip tags with BeautifulSoup. This preserves chapter boundaries, which we use as natural chunk boundaries before applying token-based splitting. One hard-won lesson: always remove page numbers and repeated header text before tokenizing the full text. Otherwise, those artifacts become part of the chunks and get translated, producing garbage in the middle of chapters. What worked: What didn't: Open question for the community: Have you found a reliable way to translate poetry, code, or tables inside long documents without breaking the surrounding narrative? We currently treat them as opaque blocks and translate them separately, but integration remains rough. Building LectuLibre taught us that long-form LLM translation is less about model capability and more about solid engineering around context management. If you're building something similar, start with robust chunking and context propagation—the model will handle the rest.