{"slug": "parsing-and-rebuilding-epub-files-in-python-lessons-learned-from-lectulibre", "title": "Parsing and Rebuilding EPUB Files in Python: Lessons Learned from LectuLibre", "summary": "LectuLibre, a service that translates books using AI, has shared the technical challenges and solutions for parsing and rebuilding EPUB files in Python. The team evaluated several libraries and settled on EbookLib for reading, lxml for HTML parsing, and Python's zipfile for rebuilding, while addressing issues like character encoding and broken manifests. They also developed a method to rebuild EPUBs directly as ZIP files to maintain fine-grained control.", "body_md": "*How we tackled EPUB parsing and rebuilding for AI book translation, with code examples and hard-won lessons.*\n\nAt LectuLibre ([https://lectulibre.com](https://lectulibre.com)), we translate books using AI. Users upload EPUB or PDF files, and our system translates the text inside while preserving the original formatting. Sounds straightforward, right? Well, we quickly learned that handling EPUB files in production is anything but trivial. In this article, I'll share the real-world challenges we faced parsing and rebuilding EPUBs with Python, the libraries we chose, the code we wrote, and the lessons we learned along the way.\n\nAn EPUB is essentially a ZIP archive containing HTML, CSS, images, and a manifest file (`content.opf`\n\n). To translate a book, we needed to:\n\nSimple in theory, but real-world EPUBs are messy. Some have broken manifests, missing required files (`mimetype`\n\n, `META-INF/container.xml`\n\n), or text encoded in obscure charsets. Others are huge, with hundreds of images that we don't want to load into memory. Our translation pipeline needed to be fast, memory-efficient, and resilient to broken input.\n\nWe evaluated several options:\n\nWe settled on EbookLib for most reading tasks, lxml for high-performance HTML parsing, and Python's built-in `zipfile`\n\nfor the final rebuilding step where we needed fine-grained control. This combination gave us the right balance of development speed and runtime performance.\n\nFirst, we read the EPUB using EbookLib and iterate over the spine (the linear reading order). We only process items of type `ITEM_DOCUMENT`\n\n(HTML files).\n\n``` python\nimport ebooklib\nfrom ebooklib import epub\n\nbook = epub.read_epub('path/to/book.epub')\n\ntranslatable_segments = []\n\nfor item in book.get_items_of_type(ebooklib.ITEM_DOCUMENT):\n    content = item.get_body_content().decode('utf-8')  # assuming utf-8\n    # But wait – not all EPUBs use utf-8!\n```\n\n**Lesson 1: Encoding Hell**\n\nMany EPUBs (especially older ones) encode their HTML in Windows-1252 or ISO-8859-1. EbookLib's `get_body_content()`\n\noften returns bytes, and decoding as utf-8 raises UnicodeDecodeError. We added charset detection using `chardet`\n\n:\n\n``` python\nimport chardet\n\nraw_bytes = item.get_content()\nencoding = chardet.detect(raw_bytes)['encoding'] or 'utf-8'\ntext = raw_bytes.decode(encoding, errors='replace')\n```\n\nOnce decoded, we parse the HTML with lxml to extract plain text while preserving the structure for later reassembly.\n\n``` python\nfrom lxml import etree\n\nparser = etree.HTMLParser()\ntree = etree.fromstring(text.encode('utf-8'), parser=parser)\n\n# Extract text with XPath, but keep track of element boundaries\nfor elem in tree.iter():\n    if elem.text:\n        translatable_segments.append({\n            'id': item.get_id(),\n            'tag': elem.tag,\n            'text': elem.text.strip(),\n            # storing path for later replacement\n            'path': tree.getpath(elem)\n        })\n```\n\nFor LLM translation, we concatenate segments into manageable chunks, preserving paragraph boundaries to give the model context.\n\nAfter translation, we need to replace the original text in the HTML documents and reassemble the EPUB. EbookLib provides `epub.write_epub()`\n\nbut we found it had a few quirks:\n\nWe opted to manipulate the EPUB as a ZIP file directly for the rebuild step, while still using EbookLib's data structures to understand the original manifest.\n\n``` python\nimport zipfile\nimport shutil\nimport os\n\ndef rebuild_epub(original_path, output_path, translated_segments):\n    # Read original EPUB as ZIP to get all files\n    with zipfile.ZipFile(original_path, 'r') as zin:\n        with zipfile.ZipFile(output_path, 'w', zipfile.ZIP_DEFLATED) as zout:\n            for item in zin.infolist():\n                if item.filename.endswith('.html') or item.filename.endswith('.xhtml'):\n                    html_content = zin.read(item.filename).decode('utf-8')\n                    # Replace translated segments using lxml\n                    tree = etree.fromstring(html_content.encode('utf-8'), parser=parser)\n                    for seg in translated_segments:\n                        if seg['id'] == item_to_id[item.filename]:\n                            elem = tree.xpath(seg['path'])[0] if seg['path'] else None\n                            if elem is not None and elem.text:\n                                elem.text = seg['translated_text']\n                    new_content = etree.tostring(tree, encoding='unicode')\n                    zout.writestr(item, new_content.encode('utf-8'))\n                else:\n                    # Copy other files (images, css, etc.) as-is\n                    zout.writestr(item, zin.read(item.filename))\n```\n\n**Lesson 2: Preserve the Mimetype File**\n\nThe EPUB specification requires that the first file in the ZIP be the `mimetype`\n\nfile, stored without compression. EbookLib does this automatically when writing, but when using zipfile manually, you must add it first with `stored`\n\ncompression. We forgot this initially and our books were rejected by strict readers. Fix:\n\n```\nzout.writestr('mimetype', 'application/epub+zip', zipfile.ZIP_STORED)\n```\n\nUsers upload terribly broken EPUBs. We learned to validate early:\n\n``` python\ndef validate_epub(file_path):\n    with zipfile.ZipFile(file_path, 'r') as zf:\n        if 'META-INF/container.xml' not in zf.namelist():\n            raise ValueError(\"Missing container.xml\")\n        if 'mimetype' not in zf.namelist():\n            raise ValueError(\"Missing mimetype file\")\n    # Also check content.opf exists and parses correctly\n```\n\nWe also check that the OPF file references all necessary resources. This catches many upload errors before they hit the expensive translation step.\n\n`lxml`\n\nand `zipfile`\n\n.`chardet`\n\nor a similar library to decode text safely.`write_epub`\n\nmight suffice. For complex rebuilds, consider manual ZIP manipulation.`mimetype`\n\nfile correctly.At LectuLibre, this EPUB processing pipeline now handles hundreds of translations per week, allowing readers around the world to enjoy books in their native language. The journey was bumpy, but the lessons we learned have made our service robust and our engineering team wiser.\n\n**What libraries or techniques have you used for EPUB processing in Python?** We'd love to hear about alternatives to EbookLib or other war stories in the comments!", "url": "https://wpnews.pro/news/parsing-and-rebuilding-epub-files-in-python-lessons-learned-from-lectulibre", "canonical_source": "https://dev.to/jacob_gong/parsing-and-rebuilding-epub-files-in-python-lessons-learned-from-lectulibre-3mgp", "published_at": "2026-08-12 03:01:56+00:00", "updated_at": "2026-08-12 03:15:04.899158+00:00", "lang": "en", "topics": ["developer-tools", "ai-products"], "entities": ["LectuLibre", "EbookLib", "lxml", "Python"], "alternates": {"html": "https://wpnews.pro/news/parsing-and-rebuilding-epub-files-in-python-lessons-learned-from-lectulibre", "markdown": "https://wpnews.pro/news/parsing-and-rebuilding-epub-files-in-python-lessons-learned-from-lectulibre.md", "text": "https://wpnews.pro/news/parsing-and-rebuilding-epub-files-in-python-lessons-learned-from-lectulibre.txt", "jsonld": "https://wpnews.pro/news/parsing-and-rebuilding-epub-files-in-python-lessons-learned-from-lectulibre.jsonld"}}