Unlimited-OCR: Parsing a 40-Page PDF in One Pass Without Your GPU Melting

Baidu released Unlimited-OCR, an MIT-licensed model that can parse dozens of pages of a PDF in a single forward pass without the GPU memory growth typical of transformer-based OCR. The model uses a novel Reference Sliding Window Attention (R-SWA) mechanism that keeps the KV cache at a fixed size, enabling constant memory and latency regardless of output length. The architecture combines DeepSeek-OCR's DeepEncoder with an MoE decoder and requires an NVIDIA GPU with at least 12 GB of VRAM.

If you have ever built a document-parsing pipeline, you know the ritual. Split the PDF into pages. Run each page through the OCR model. Stitch the outputs back together. Then write a pile of glue code to fix the tables that got cut in half at a page boundary, and the heading that lost its section, and the footnote that ended up orphaned three pages away from its reference. Baidu's Unlimited-OCR https://github.com/baidu/Unlimited-OCR is a bet that you should not have to do any of that. It parses dozens of pages in a single forward pass, and it is MIT licensed. This post covers what the model actually does differently, and then three concrete ways to get it running: a quick Transformers script, an SGLang server for real throughput, and a Docker image if you would rather not touch a Python environment at all. Standard transformer decoding has a cost that most OCR benchmarks quietly hide: the KV cache grows with every token you generate. The KV cache is the model's short-term memory. Every token the model produces gets appended to it so future tokens can attend backwards. For a chatbot writing three paragraphs, this is fine. For an OCR model transcribing a 40-page technical manual into 30,000 tokens of Markdown, it is not. Memory climbs, attention cost climbs with it, and generation gets slower the longer it runs. Somewhere around page eight your throughput graph stops looking like a line and starts looking like a cliff. The industry workaround has been chunking. Process one page, dump the cache, process the next page, and accept that the model has no idea what came before. Unlimited-OCR attacks the cache growth directly. The team replaced every attention layer in the decoder with something they call Reference Sliding Window Attention R-SWA . The intuition is close to how a human copy-typist works. You do not hold every word you have already typed in your head. You hold the last sentence or two, and you keep glancing back at the source document. R-SWA does the same thing: the decoder keeps a fixed-size window of recently generated tokens, but it retains permanent access to the original image tokens. The KV cache is implemented as a queue with a fixed capacity, so when a new token arrives, the oldest one in the window gets evicted. The result is a cache size that is constant rather than growing. Memory and per-token latency stay flat whether you are on token 500 or token 30,000. The architecture is DeepSeek-OCR's DeepEncoder SAM-ViT-B plus CLIP-L feeding an MoE decoder. The encoder's aggressive visual token compression is what keeps the image side of the cache small enough for the whole thing to work. One thing the paper flags that is easy to miss: R-SWA is not OCR-specific. It is a general-purpose attention mechanism for any long-horizon transcription-shaped task, ASR included. You need an NVIDIA GPU. There is no CPU or Apple Silicon path in the official repo. The weights are bfloat16, so roughly 6 GB just to load the model, before activations and image tokens. A 12 GB card is a sane floor for single-image work; for multi-page runs at 32K context you will want more headroom. The maintainers tested on Python 3.12.3 with CUDA 12.9. If you just want to see output before committing to any of this, there is a Hugging Face Space https://huggingface.co/spaces/baidu/Unlimited-OCR you can drop a file into. This is the fastest way to a working result. Set up the environment: python -m venv .venv source .venv/bin/activate pip install torch==2.10.0 torchvision==0.25.0 transformers==4.57.1 pip install Pillow==12.1.1 matplotlib==3.10.8 einops==0.8.2 pip install addict==2.4.0 easydict==1.13 pymupdf==1.27.2.2 psutil==7.2.2 Pin these versions. The model ships custom modeling code via trust remote code , and that code is written against these specific releases. Version drift here produces confusing import errors rather than clean failures. Now a single image: python import torch from transformers import AutoModel, AutoTokenizer model name = 'baidu/Unlimited-OCR' tokenizer = AutoTokenizer.from pretrained model name, trust remote code=True model = AutoModel.from pretrained model name, trust remote code=True, use safetensors=True, torch dtype=torch.bfloat16, model = model.eval .cuda model.infer tokenizer, prompt='<image document parsing.', image file='your image.jpg', output path='your/output/dir', base size=1024, image size=640, crop mode=True, gundam mode max length=32768, no repeat ngram size=35, ngram window=128, save results=True, Two details in there matter more than they look: The prompt must literally start with <image . It is not decoration, it is the placeholder token the visual features get spliced into. Drop it and you will get output that looks like the model hallucinating a document it never saw. no repeat ngram size and ngram window are load-bearing. Long-horizon generation on repetitive layouts, think a table of contents or a price list, can send the model into a loop where it happily emits the same row forever. These parameters block any 35-gram from repeating inside a sliding window. Do not remove them because they look like tuning noise. Single-image inference gives you two configurations, and the naming is not self-explanatory: | Mode | Settings | Use for | |---|---|---| | gundam | base size=1024, image size=640, crop mode=True | Single images, especially dense ones. Crops the image into tiles and processes each, so small text survives. | | base | base size=1024, image size=1024, crop mode=False | Whole image at once. Required for multi-page. | Multi-page and PDF paths only support base mode. That is a hard constraint, not a default. model.infer multi tokenizer, prompt='<image Multi page parsing.', image files= 'page1.png', 'page2.png', 'page3.png' , output path='your/output/dir', image size=1024, max length=32768, no repeat ngram size=35, ngram window=1024, save results=True, Note that ngram window jumps from 128 to 1024 here. Across many pages there is legitimately more repeated structure, so the repeat-detection window has to widen to keep up. There is no direct PDF entry point. You rasterize first, then feed the images to infer multi : python import os, tempfile import fitz PyMuPDF def pdf to images pdf path, dpi=300 : doc = fitz.open pdf path tmp dir = tempfile.mkdtemp prefix='pdf ocr ' mat = fitz.Matrix dpi / 72, dpi / 72 paths = for i, page in enumerate doc : out = os.path.join tmp dir, f'page {i+1:04d}.png' page.get pixmap matrix=mat .save out paths.append out doc.close return paths model.infer multi tokenizer, prompt='<image Multi page parsing.', image files=pdf to images 'your doc.pdf', dpi=300 , output path='your/output/dir', image size=1024, max length=32768, no repeat ngram size=35, ngram window=1024, save results=True, 300 DPI is the recommended default. Going lower to save memory costs you small text and table rules, which is usually the exact content you cared about. Transformers is fine for evaluating the model. For a service handling concurrent requests, run SGLang and talk to it over an OpenAI-compatible API. Set up the environment. Note that SGLang ships as a local wheel in the repo rather than from PyPI: uv venv --python 3.12 source .venv/bin/activate uv pip install wheel/sglang-0.0.0.dev11416+g92e8bb79e-py3-none-any.whl uv pip install kernels==0.11.7 uv pip install pymupdf==1.27.2.2 Small heads-up: the README prose mentions pinning kernels==0.9.0 while the command block right beneath it installs 0.11.7 . Follow the command block. If you hit kernel-related errors, that mismatch is the first thing to check against the current repo state. Launch the server: python -m sglang.launch server \ --model baidu/Unlimited-OCR \ --served-model-name Unlimited-OCR \ --attention-backend fa3 \ --page-size 1 \ --mem-fraction-static 0.8 \ --context-length 32768 \ --enable-custom-logit-processor \ --disable-overlap-schedule \ --skip-server-warmup \ --host 0.0.0.0 \ --port 10000 The flags that are not optional: --enable-custom-logit-processor — the no-repeat-ngram processor runs as a custom logit processor. Without this flag, requests referencing it fail. --attention-backend fa3 — FlashAttention 3, which requires Hopper-class hardware. On older GPUs you will need a different backend. --page-size 1 and --disable-overlap-schedule — R-SWA's cache eviction does not play nicely with the usual paged-attention and overlapped-scheduling optimizations.Then a client. The image goes in as a base64 data URL, standard OpenAI vision format, with a couple of model-specific extras: python import base64, json, os import requests from sglang.srt.sampling.custom logit processor import DeepseekOCRNoRepeatNGramLogitProcessor, server url = "http://127.0.0.1:10000" session = requests.Session session.trust env = False def encode image image path : ext = os.path.splitext image path 1 .lower mime = "image/jpeg" if ext in ".jpg", ".jpeg" else f"image/{ext.lstrip '.' }" with open image path, "rb" as f: data = base64.b64encode f.read .decode "utf-8" return {"type": "image url", "image url": {"url": f"data:{mime};base64,{data}"}} def generate prompt, image paths, image mode, ngram window : content = {"type": "text", "text": prompt} + encode image p for p in image paths payload = { "model": "Unlimited-OCR", "messages": {"role": "user", "content": content} , "temperature": 0, "skip special tokens": False, "images config": {"image mode": image mode}, "custom logit processor": DeepseekOCRNoRepeatNGramLogitProcessor.to str , "custom params": {"ngram size": 35, "window size": ngram window}, "stream": True, } response = session.post f"{server url}/v1/chat/completions", headers={"Content-Type": "application/json"}, data=json.dumps payload , timeout=1200, stream=True, response.raise for status chunks = for line in response.iter lines chunk size=1, decode unicode=True : if not line or not line.startswith "data: " : continue data = line len "data: " : if data == " DONE ": break delta = json.loads data "choices" 0 .get "delta", {} .get "content", "" if delta: print delta, end="", flush=True chunks.append delta return "".join chunks generate "document parsing.", "your image.jpg" , image mode="gundam", ngram window=128 temperature: 0 and skip special tokens: False are both deliberate. This is transcription, not generation, so any sampling randomness is pure downside. And the special tokens carry layout structure you want in the output. The 1200 second timeout is also not paranoia. Long documents take a while, which is precisely why you want streaming. The repo ships infer.py , which starts the SGLang server for you and fires concurrent requests at it: A directory of images python infer.py \ --image dir ./examples/images \ --output dir ./outputs \ --concurrency 8 \ --image mode gundam A PDF python infer.py \ --pdf ./examples/document.pdf \ --output dir ./outputs \ --concurrency 8 \ --image mode gundam Useful extras: --model dir accepts a local path or a Hugging Face ID, --gpu sets CUDA VISIBLE DEVICES , and --server log puts server output somewhere you can read it. Start concurrency low. Eight is the documented example, but the right number depends on your VRAM and how long your documents are. If you want to skip environment management entirely: Default, CUDA 13.0 docker pull vllm/vllm-openai:unlimited-ocr Hopper GPUs, CUDA 12.9 docker pull vllm/vllm-openai:unlimited-ocr-cu129 docker run --rm --gpus all --network host --ipc host \ vllm/vllm-openai:unlimited-ocr \ baidu/Unlimited-OCR \ --trust-remote-code \ --logits processors vllm.model executor.models.unlimited ocr:NGramPerReqLogitsProcessor \ --no-enable-prefix-caching \ --mm-processor-cache-gb 0 Same story as SGLang regarding the ngram processor, it has to be registered at server start. Prefix caching and the multimodal processor cache are both disabled because R-SWA's fixed-window cache invalidates the assumptions those optimizations make. Full details are in the official vLLM recipe https://recipes.vllm.ai/baidu/Unlimited-OCR . | Parameter | Single image | Multi-page / PDF | |---|---|---| | mode | gundam or base | base only | image size | 640 gundam / 1024 base | 1024 | crop mode | True gundam / False base | False | ngram window | 128 | 1024 | no repeat ngram size | 35 | 35 | max length | 32768 | 32768 | | prompt | <image document parsing. | <image Multi page parsing. | Reach for something else if you need bounding boxes and per-word confidence scores, since this is an end-to-end model producing Markdown, not a detection-plus-recognition pipeline. Same if you are OCRing short receipts or single lines, where a 3B model on a GPU is enormous overkill compared to PaddleOCR or Tesseract. And if you have no NVIDIA GPU, there is no supported path today. The interesting claim here is not the benchmark number. It is that a fixed-size KV cache made the model both faster and more accurate on long documents, when efficiency work almost always costs you quality somewhere. If R-SWA generalizes the way the authors suggest, the same trick applies to any task where a model transcribes a long input into a long output while keeping the source in view. Long-form ASR is the obvious next one.