# Unlimited-OCR: Parsing a 40-Page PDF in One Pass Without Your GPU Melting > Source: > Published: 2026-07-24 17:19:34+00:00 If you have ever built a document-parsing pipeline, you know the ritual. Split the PDF into pages. Run each page through the OCR model. Stitch the outputs back together. Then write a pile of glue code to fix the tables that got cut in half at a page boundary, and the heading that lost its section, and the footnote that ended up orphaned three pages away from its reference. Baidu's [Unlimited-OCR](https://github.com/baidu/Unlimited-OCR) is a bet that you should not have to do any of that. It parses dozens of pages in a single forward pass, and it is MIT licensed. This post covers what the model actually does differently, and then three concrete ways to get it running: a quick Transformers script, an SGLang server for real throughput, and a Docker image if you would rather not touch a Python environment at all. Standard transformer decoding has a cost that most OCR benchmarks quietly hide: the KV cache grows with every token you generate. The KV cache is the model's short-term memory. Every token the model produces gets appended to it so future tokens can attend backwards. For a chatbot writing three paragraphs, this is fine. For an OCR model transcribing a 40-page technical manual into 30,000 tokens of Markdown, it is not. Memory climbs, attention cost climbs with it, and generation gets slower the longer it runs. Somewhere around page eight your throughput graph stops looking like a line and starts looking like a cliff. The industry workaround has been chunking. Process one page, dump the cache, process the next page, and accept that the model has no idea what came before. Unlimited-OCR attacks the cache growth directly. The team replaced every attention layer in the decoder with something they call **Reference Sliding Window Attention (R-SWA)**. The intuition is close to how a human copy-typist works. You do not hold every word you have already typed in your head. You hold the last sentence or two, and you keep glancing back at the source document. R-SWA does the same thing: the decoder keeps a fixed-size window of recently generated tokens, but it retains permanent access to the original image tokens. The KV cache is implemented as a queue with a fixed capacity, so when a new token arrives, the oldest one in the window gets evicted. The result is a cache size that is constant rather than growing. Memory and per-token latency stay flat whether you are on token 500 or token 30,000. The architecture is DeepSeek-OCR's DeepEncoder (SAM-ViT-B plus CLIP-L) feeding an MoE decoder. The encoder's aggressive visual token compression is what keeps the image side of the cache small enough for the whole thing to work. One thing the paper flags that is easy to miss: R-SWA is not OCR-specific. It is a general-purpose attention mechanism for any long-horizon transcription-shaped task, ASR included. You need an NVIDIA GPU. There is no CPU or Apple Silicon path in the official repo. The weights are bfloat16, so roughly 6 GB just to load the model, before activations and image tokens. A 12 GB card is a sane floor for single-image work; for multi-page runs at 32K context you will want more headroom. The maintainers tested on Python 3.12.3 with CUDA 12.9. If you just want to see output before committing to any of this, there is a [Hugging Face Space](https://huggingface.co/spaces/baidu/Unlimited-OCR) you can drop a file into. This is the fastest way to a working result. Set up the environment: ``` python -m venv .venv source .venv/bin/activate pip install torch==2.10.0 torchvision==0.25.0 transformers==4.57.1 pip install Pillow==12.1.1 matplotlib==3.10.8 einops==0.8.2 pip install addict==2.4.0 easydict==1.13 pymupdf==1.27.2.2 psutil==7.2.2 ``` Pin these versions. The model ships custom modeling code via `trust_remote_code` , and that code is written against these specific releases. Version drift here produces confusing import errors rather than clean failures. Now a single image: ``` python import torch from transformers import AutoModel, AutoTokenizer model_name = 'baidu/Unlimited-OCR' tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) model = AutoModel.from_pretrained( model_name, trust_remote_code=True, use_safetensors=True, torch_dtype=torch.bfloat16, ) model = model.eval().cuda() model.infer( tokenizer, prompt='document parsing.', image_file='your_image.jpg', output_path='your/output/dir', base_size=1024, image_size=640, crop_mode=True, # gundam mode max_length=32768, no_repeat_ngram_size=35, ngram_window=128, save_results=True, ) ``` Two details in there matter more than they look: **The prompt must literally start with .** It is not decoration, it is the placeholder token the visual features get spliced into. Drop it and you will get output that looks like the model hallucinating a document it never saw. ** no_repeat_ngram_size and ngram_window are load-bearing.** Long-horizon generation on repetitive layouts, think a table of contents or a price list, can send the model into a loop where it happily emits the same row forever. These parameters block any 35-gram from repeating inside a sliding window. Do not remove them because they look like tuning noise. Single-image inference gives you two configurations, and the naming is not self-explanatory: | Mode | Settings | Use for | |---|---|---| | gundam | `base_size=1024, image_size=640, crop_mode=True` | Single images, especially dense ones. Crops the image into tiles and processes each, so small text survives. | | base | `base_size=1024, image_size=1024, crop_mode=False` | Whole image at once. Required for multi-page. | Multi-page and PDF paths only support base mode. That is a hard constraint, not a default. ``` model.infer_multi( tokenizer, prompt='Multi page parsing.', image_files=['page1.png', 'page2.png', 'page3.png'], output_path='your/output/dir', image_size=1024, max_length=32768, no_repeat_ngram_size=35, ngram_window=1024, save_results=True, ) ``` Note that `ngram_window` jumps from 128 to 1024 here. Across many pages there is legitimately more repeated structure, so the repeat-detection window has to widen to keep up. There is no direct PDF entry point. You rasterize first, then feed the images to `infer_multi` : ``` python import os, tempfile import fitz # PyMuPDF def pdf_to_images(pdf_path, dpi=300): doc = fitz.open(pdf_path) tmp_dir = tempfile.mkdtemp(prefix='pdf_ocr_') mat = fitz.Matrix(dpi / 72, dpi / 72) paths = [] for i, page in enumerate(doc): out = os.path.join(tmp_dir, f'page_{i+1:04d}.png') page.get_pixmap(matrix=mat).save(out) paths.append(out) doc.close() return paths model.infer_multi( tokenizer, prompt='Multi page parsing.', image_files=pdf_to_images('your_doc.pdf', dpi=300), output_path='your/output/dir', image_size=1024, max_length=32768, no_repeat_ngram_size=35, ngram_window=1024, save_results=True, ) ``` 300 DPI is the recommended default. Going lower to save memory costs you small text and table rules, which is usually the exact content you cared about. Transformers is fine for evaluating the model. For a service handling concurrent requests, run SGLang and talk to it over an OpenAI-compatible API. Set up the environment. Note that SGLang ships as a local wheel in the repo rather than from PyPI: ``` uv venv --python 3.12 source .venv/bin/activate uv pip install wheel/sglang-0.0.0.dev11416+g92e8bb79e-py3-none-any.whl uv pip install kernels==0.11.7 uv pip install pymupdf==1.27.2.2 ``` Small heads-up: the README prose mentions pinning `kernels==0.9.0` while the command block right beneath it installs `0.11.7` . Follow the command block. If you hit kernel-related errors, that mismatch is the first thing to check against the current repo state. Launch the server: ``` python -m sglang.launch_server \ --model baidu/Unlimited-OCR \ --served-model-name Unlimited-OCR \ --attention-backend fa3 \ --page-size 1 \ --mem-fraction-static 0.8 \ --context-length 32768 \ --enable-custom-logit-processor \ --disable-overlap-schedule \ --skip-server-warmup \ --host 0.0.0.0 \ --port 10000 ``` The flags that are not optional: `--enable-custom-logit-processor` — the no-repeat-ngram processor runs as a custom logit processor. Without this flag, requests referencing it fail.`--attention-backend fa3` — FlashAttention 3, which requires Hopper-class hardware. On older GPUs you will need a different backend.`--page-size 1` and `--disable-overlap-schedule` — R-SWA's cache eviction does not play nicely with the usual paged-attention and overlapped-scheduling optimizations.Then a client. The image goes in as a base64 data URL, standard OpenAI vision format, with a couple of model-specific extras: ``` python import base64, json, os import requests from sglang.srt.sampling.custom_logit_processor import ( DeepseekOCRNoRepeatNGramLogitProcessor, ) server_url = "http://127.0.0.1:10000" session = requests.Session() session.trust_env = False def encode_image(image_path): ext = os.path.splitext(image_path)[1].lower() mime = "image/jpeg" if ext in (".jpg", ".jpeg") else f"image/{ext.lstrip('.')}" with open(image_path, "rb") as f: data = base64.b64encode(f.read()).decode("utf-8") return {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{data}"}} def generate(prompt, image_paths, image_mode, ngram_window): content = [{"type": "text", "text": prompt}] + [ encode_image(p) for p in image_paths ] payload = { "model": "Unlimited-OCR", "messages": [{"role": "user", "content": content}], "temperature": 0, "skip_special_tokens": False, "images_config": {"image_mode": image_mode}, "custom_logit_processor": DeepseekOCRNoRepeatNGramLogitProcessor.to_str(), "custom_params": {"ngram_size": 35, "window_size": ngram_window}, "stream": True, } response = session.post( f"{server_url}/v1/chat/completions", headers={"Content-Type": "application/json"}, data=json.dumps(payload), timeout=1200, stream=True, ) response.raise_for_status() chunks = [] for line in response.iter_lines(chunk_size=1, decode_unicode=True): if not line or not line.startswith("data: "): continue data = line[len("data: "):] if data == "[DONE]": break delta = json.loads(data)["choices"][0].get("delta", {}).get("content", "") if delta: print(delta, end="", flush=True) chunks.append(delta) return "".join(chunks) generate("document parsing.", ["your_image.jpg"], image_mode="gundam", ngram_window=128) ``` `temperature: 0` and `skip_special_tokens: False` are both deliberate. This is transcription, not generation, so any sampling randomness is pure downside. And the special tokens carry layout structure you want in the output. The 1200 second timeout is also not paranoia. Long documents take a while, which is precisely why you want streaming. The repo ships `infer.py` , which starts the SGLang server for you and fires concurrent requests at it: ``` # A directory of images python infer.py \ --image_dir ./examples/images \ --output_dir ./outputs \ --concurrency 8 \ --image_mode gundam # A PDF python infer.py \ --pdf ./examples/document.pdf \ --output_dir ./outputs \ --concurrency 8 \ --image_mode gundam ``` Useful extras: `--model_dir` accepts a local path or a Hugging Face ID, `--gpu` sets `CUDA_VISIBLE_DEVICES` , and `--server_log` puts server output somewhere you can read it. Start concurrency low. Eight is the documented example, but the right number depends on your VRAM and how long your documents are. If you want to skip environment management entirely: ``` # Default, CUDA 13.0 docker pull vllm/vllm-openai:unlimited-ocr # Hopper GPUs, CUDA 12.9 docker pull vllm/vllm-openai:unlimited-ocr-cu129 docker run --rm --gpus all --network host --ipc host \ vllm/vllm-openai:unlimited-ocr \ baidu/Unlimited-OCR \ --trust-remote-code \ --logits_processors vllm.model_executor.models.unlimited_ocr:NGramPerReqLogitsProcessor \ --no-enable-prefix-caching \ --mm-processor-cache-gb 0 ``` Same story as SGLang regarding the ngram processor, it has to be registered at server start. Prefix caching and the multimodal processor cache are both disabled because R-SWA's fixed-window cache invalidates the assumptions those optimizations make. Full details are in the [official vLLM recipe](https://recipes.vllm.ai/baidu/Unlimited-OCR). | Parameter | Single image | Multi-page / PDF | |---|---|---| | mode | gundam or base | base only | `image_size` | 640 (gundam) / 1024 (base) | 1024 | `crop_mode` | True (gundam) / False (base) | False | `ngram_window` | 128 | 1024 | `no_repeat_ngram_size` | 35 | 35 | `max_length` | 32768 | 32768 | | prompt | `document parsing.` | `Multi page parsing.` | Reach for something else if you need bounding boxes and per-word confidence scores, since this is an end-to-end model producing Markdown, not a detection-plus-recognition pipeline. Same if you are OCRing short receipts or single lines, where a 3B model on a GPU is enormous overkill compared to PaddleOCR or Tesseract. And if you have no NVIDIA GPU, there is no supported path today. The interesting claim here is not the benchmark number. It is that a fixed-size KV cache made the model both faster and more accurate on long documents, when efficiency work almost always costs you quality somewhere. If R-SWA generalizes the way the authors suggest, the same trick applies to any task where a model transcribes a long input into a long output while keeping the source in view. Long-form ASR is the obvious next one.