{"slug": "building-a-custom-ai-pdf-reader-in-python-from-a-jupyter-prototype-to-tested", "title": "Building a Custom AI PDF Reader in Python: From a Jupyter Prototype to Tested Modules", "summary": "A developer built a custom AI PDF reader in Python, starting with a Jupyter prototype and evolving it into tested modules. The reader uses PyMuPDF for rendering and search, with features like bookmarks, notes, and zoom. The project highlights the importance of separating PDF coordinates from image coordinates and using a centralized state for UI consistency.", "body_md": "Research papers are much easier to read when the reader fits the way you work. I wanted a PDF reader that could eventually support bookmarks, notes, annotations, summaries, question answering, text-to-speech, and voice commands.\n\nRather than trying to write a complete PDF engine from scratch, I started with a small Python prototype. The goal was simple: learn each layer of the application properly, build a working foundation, and only then move toward a desktop application.\n\nThis article documents the first stage of that journey: an interactive PDF reader in Jupyter, local persistence for bookmarks and notes, a cleaner module structure, automated tests, and the lessons learned along the way.\n\nThe long-term goal is a customizable desktop PDF reader for research reading. The eventual application may include:\n\nFor the first milestone, I deliberately kept the scope smaller. I focused on PDF rendering, navigation, search, persistence, and tests.\n\nI chose Python because it allowed me to experiment quickly. My first stack was:\n\n| Need | Tool |\n|---|---|\n| PDF rendering, text extraction, and search | PyMuPDF |\n| Interactive prototype interface | JupyterLab and ipywidgets |\n| Image handling | Pillow |\n| Local saved data | JSON |\n| Version control | Git and GitHub |\n| Automated tests | pytest |\n\nThe future desktop UI will use PySide6, but Jupyter was a useful place to learn the reader logic before dealing with desktop-window layouts, signals, menus, and packaging.\n\nThe prototype opens a local PDF with PyMuPDF:\n\n``` python\nfrom pathlib import Path\nimport pymupdf\n\nPDF_PATH = Path(\"test.pdf\")\ndocument = pymupdf.open(PDF_PATH)\n\nprint(document.page_count)\n```\n\nA PDF page is not automatically an image. PyMuPDF renders a page into a pixmap, and Pillow converts the pixel data into an image that Jupyter can display.\n\n``` python\nfrom PIL import Image\n\npage = document[0]\npixmap = page.get_pixmap(dpi=120)\n\nimage = Image.frombytes(\n    \"RGB\",\n    (pixmap.width, pixmap.height),\n    pixmap.samples\n)\n\nimage\n```\n\nThis gave me the basic page view. From there, I added buttons for Previous, Next, Go to page, Zoom In, and Zoom Out.\n\nOne important concept I learned was application state. Instead of letting each button manage unrelated variables, I kept the reader's current information together:\n\n```\nreader_state = {\n    \"current_page\": 0,\n    \"zoom_dpi\": 120,\n    \"search_results\": [],\n    \"search_index\": 0,\n    \"bookmarks\": [],\n    \"notes\": []\n}\n```\n\nThe UI follows a simple pattern:\n\n`reader_state`\n\n.For example, page navigation uses one shared function:\n\n``` python\ndef change_page(new_page):\n    if not 0 <= new_page < document.page_count:\n        return\n\n    reader_state[\"current_page\"] = new_page\n    save_current_reader_data()\n    refresh_reader()\n```\n\nUsing a single function for navigation prevents different controls from handling page changes in slightly different ways.\n\nPyMuPDF can find the rectangles where a text query appears on a page.\n\n```\nrectangles = page.search_for(\"research\")\n```\n\nThe returned rectangles use PDF coordinates, measured in points. The rendered page image uses pixels. Since PDF points are based on 72 points per inch, I learned to scale each search rectangle with this formula:\n\n```\nscale = dpi / 72\npixel_x = pdf_x * scale\n```\n\nThat conversion lets the reader draw highlights in the correct position on the rendered image.\n\n``` python\nfrom PIL import ImageDraw\n\ndraw = ImageDraw.Draw(image, \"RGBA\")\n\nfor rect in rectangles:\n    draw.rectangle(\n        [\n            rect.x0 * scale,\n            rect.y0 * scale,\n            rect.x1 * scale,\n            rect.y1 * scale\n        ],\n        fill=(255, 235, 0, 90),\n        outline=(255, 0, 0, 255),\n        width=3\n    )\n```\n\nThis was one of the most useful lessons in the project: PDF document coordinates and screen-image coordinates are not the same thing.\n\nBookmarks and notes should survive a restart. I did not want to modify the original PDF for this first version, so I stored personal reader data in a JSON file.\n\nA document entry looks like this:\n\n```\n{\n  \"documents\": {\n    \"/absolute/path/to/test.pdf\": {\n      \"file_name\": \"test.pdf\",\n      \"bookmarks\": [\n        {\n          \"page_number\": 2,\n          \"label\": \"Important result\"\n        }\n      ],\n      \"notes\": [\n        {\n          \"page_number\": 5,\n          \"text\": \"Review this figure before the presentation.\",\n          \"created_at\": \"2026-08-11 15:30\"\n        }\n      ],\n      \"last_page\": 5\n    }\n  }\n}\n```\n\nA bookmark points to a page and has a label. A note points to a page, has text, and stores when it was created. The current page is also saved whenever the user navigates.\n\nI used a temporary file before replacing the main JSON file:\n\n``` python\ndef save_all_reader_data(data):\n    temporary_path = DATA_PATH.with_suffix(\".tmp\")\n\n    with temporary_path.open(\"w\", encoding=\"utf-8\") as file:\n        json.dump(data, file, indent=2, ensure_ascii=False)\n\n    os.replace(temporary_path, DATA_PATH)\n```\n\nThis is safer than directly overwriting the main file because it reduces the risk of leaving a partially written JSON file if a save is interrupted.\n\nThe first notebook worked, but it was becoming one large cell. That is acceptable for exploration, but difficult to maintain.\n\nI moved reusable code into separate modules:\n\n```\ncustom-ai-pdf-reader/\n├── data/\n│   └── reader_data.json\n├── notebooks/\n│   └── Untitled.ipynb\n├── src/\n│   ├── __init__.py\n│   ├── pdf_service.py\n│   ├── reader_state.py\n│   └── storage_service.py\n├── tests/\n│   └── test_storage.py\n├── .gitignore\n├── pytest.ini\n└── requirements.txt\n```\n\n`pdf_service.py`\n\nThis module is responsible for PDF-specific tasks:\n\n``` python\ndef open_pdf(pdf_path):\n    ...\n\ndef get_page_count(document):\n    ...\n\ndef render_page(document, page_number, dpi, highlight_rectangles=None):\n    ...\n\ndef search_document(document, query):\n    ...\n```\n\n`storage_service.py`\n\nThis module handles JSON persistence:\n\n``` python\ndef load_all_reader_data():\n    ...\n\ndef save_all_reader_data(data):\n    ...\n\ndef load_document_data(document_id, pdf_path):\n    ...\n\ndef save_document_data(document_id, pdf_path, bookmarks, notes, last_page):\n    ...\n```\n\n`reader_state.py`\n\nThis module stores the reader's active state:\n\n```\nreader_state = {\n    \"current_page\": 0,\n    \"zoom_dpi\": 120,\n    \"search_results\": [],\n    \"search_index\": 0,\n    \"bookmarks\": [],\n    \"notes\": []\n}\n```\n\nThe notebook now focuses on the interface and event handlers, while the reusable logic lives in Python files.\n\nI added pytest tests for the storage layer. The tests use temporary folders, so they do not touch my real bookmarks, notes, or `reader_data.json`\n\nfile.\n\nExamples of what the tests verify:\n\nThe first successful test run was a good milestone:\n\n```\ncollected 5 items\n\n5 passed in 0.02s\n```\n\nThis was also my first practical lesson in why automated tests matter. The interface can look correct while a save or load function still has a hidden problem. Tests give the project a repeatable safety net before making larger changes.\n\nBuilding this project involved several useful mistakes and fixes.\n\nI initially considered a Tkinter desktop interface, but the Linux Python environment did not include the required Tk bindings. Instead of spending the first phase on GUI installation problems, I switched to Jupyter widgets for the prototype and chose PySide6 for the future desktop application.\n\nAn Ubuntu background update held the package-manager lock. The correct response was to wait and inspect the running update, not to delete lock files or force-stop the process.\n\nAt one stage, interactive Jupyter widgets appeared as plain text or did not respond to clicks. The problem was environment setup: Jupyter, the Python kernel, and ipywidgets need to be connected to the same project environment. Restarting the kernel and testing a minimal button helped isolate the issue.\n\nWhen I moved code into modules, I accidentally kept old versions of functions such as `render_page()`\n\n, `search_document()`\n\n, and JSON storage functions inside the notebook. This created duplicate names and confusing behavior. The fix was to keep the reusable function in one module and import it into the notebook.\n\nI created a new project folder inside an existing Git repository and accidentally ran `git init`\n\ninside the nested folder. That created a second `.git`\n\ndirectory. The correct approach was to remove only the accidental nested `.git`\n\nfolder and use the original repository at the parent level.\n\n`src`\n\nMy tests initially failed with `ModuleNotFoundError: No module named 'src'`\n\n. I fixed that by adding `src/__init__.py`\n\nand configuring pytest with a `pytest.ini`\n\nfile.\n\n```\n[pytest]\npythonpath = .\ntestpaths = tests\n```\n\nThe tests then passed.\n\nI learned to use a safer Git workflow for each tested milestone:\n\n```\ngit status\ngit add custom-ai-pdf-reader/src/\ngit add custom-ai-pdf-reader/tests/\ngit add custom-ai-pdf-reader/notebooks/\ngit commit -m \"Add modular PDF reader prototype and storage tests\"\ngit push\n```\n\nThe `.gitignore`\n\nfile is important because local PDFs, the virtual environment, temporary cache files, and personal reader data should not be uploaded.\n\n```\n__pycache__/\n*.py[cod]\n.pytest_cache/\n.venv/\n.ipynb_checkpoints/\ndata/\n*.pdf\n```\n\nThis project taught me more than how to display a PDF in Python. The main lessons were:\n\nThe Jupyter prototype now has a tested storage layer. The next milestone is a PySide6 desktop MVP with:\n\nAfter the desktop foundation is stable, I plan to migrate persistence from JSON to SQLite, add text selection and annotations, then explore AI summaries, PDF question answering with citations, text-to-speech, and voice commands.\n\nStarting small was the right choice. I did not begin by trying to build an AI system or a complete commercial PDF application. I started with one page, then navigation, then search, then bookmarks, notes, saved progress, modules, tests, and version control.\n\nEach feature made the next one easier to understand. The project is still at an early stage, but it already has a working and tested foundation for a more capable custom research reader.\n\nIf you are building your own developer project, my advice is simple: keep the first version small, make each milestone testable, and commit working progress often.", "url": "https://wpnews.pro/news/building-a-custom-ai-pdf-reader-in-python-from-a-jupyter-prototype-to-tested", "canonical_source": "https://dev.to/kavindu_kp/building-a-custom-ai-pdf-reader-in-python-from-a-jupyter-prototype-to-tested-modules-58ld", "published_at": "2026-08-14 16:11:33+00:00", "updated_at": "2026-08-14 16:35:24.175223+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence"], "entities": ["PyMuPDF", "JupyterLab", "ipywidgets", "Pillow", "pytest", "Git", "GitHub", "PySide6"], "alternates": {"html": "https://wpnews.pro/news/building-a-custom-ai-pdf-reader-in-python-from-a-jupyter-prototype-to-tested", "markdown": "https://wpnews.pro/news/building-a-custom-ai-pdf-reader-in-python-from-a-jupyter-prototype-to-tested.md", "text": "https://wpnews.pro/news/building-a-custom-ai-pdf-reader-in-python-from-a-jupyter-prototype-to-tested.txt", "jsonld": "https://wpnews.pro/news/building-a-custom-ai-pdf-reader-in-python-from-a-jupyter-prototype-to-tested.jsonld"}}