{"slug": "unlimited-ocr-parsing-a-40-page-pdf-in-one-pass-without-your-gpu-melting", "title": "Unlimited-OCR: Parsing a 40-Page PDF in One Pass Without Your GPU Melting", "summary": "Baidu released Unlimited-OCR, an MIT-licensed model that can parse dozens of pages of a PDF in a single forward pass without the GPU memory growth typical of transformer-based OCR. The model uses a novel Reference Sliding Window Attention (R-SWA) mechanism that keeps the KV cache at a fixed size, enabling constant memory and latency regardless of output length. The architecture combines DeepSeek-OCR's DeepEncoder with an MoE decoder and requires an NVIDIA GPU with at least 12 GB of VRAM.", "body_md": "If you have ever built a document-parsing pipeline, you know the ritual. Split the PDF into pages. Run each page through the OCR model. Stitch the outputs back together. Then write a pile of glue code to fix the tables that got cut in half at a page boundary, and the heading that lost its section, and the footnote that ended up orphaned three pages away from its reference.\n\nBaidu's [Unlimited-OCR](https://github.com/baidu/Unlimited-OCR) is a bet that you should not have to do any of that. It parses dozens of pages in a single forward pass, and it is MIT licensed.\n\nThis post covers what the model actually does differently, and then three concrete ways to get it running: a quick Transformers script, an SGLang server for real throughput, and a Docker image if you would rather not touch a Python environment at all.\n\nStandard transformer decoding has a cost that most OCR benchmarks quietly hide: the KV cache grows with every token you generate.\n\nThe KV cache is the model's short-term memory. Every token the model produces gets appended to it so future tokens can attend backwards. For a chatbot writing three paragraphs, this is fine. For an OCR model transcribing a 40-page technical manual into 30,000 tokens of Markdown, it is not. Memory climbs, attention cost climbs with it, and generation gets slower the longer it runs. Somewhere around page eight your throughput graph stops looking like a line and starts looking like a cliff.\n\nThe industry workaround has been chunking. Process one page, dump the cache, process the next page, and accept that the model has no idea what came before.\n\nUnlimited-OCR attacks the cache growth directly. The team replaced every attention layer in the decoder with something they call **Reference Sliding Window Attention (R-SWA)**.\n\nThe intuition is close to how a human copy-typist works. You do not hold every word you have already typed in your head. You hold the last sentence or two, and you keep glancing back at the source document. R-SWA does the same thing: the decoder keeps a fixed-size window of recently generated tokens, but it retains permanent access to the original image tokens. The KV cache is implemented as a queue with a fixed capacity, so when a new token arrives, the oldest one in the window gets evicted.\n\nThe result is a cache size that is constant rather than growing. Memory and per-token latency stay flat whether you are on token 500 or token 30,000.\n\nThe architecture is DeepSeek-OCR's DeepEncoder (SAM-ViT-B plus CLIP-L) feeding an MoE decoder. The encoder's aggressive visual token compression is what keeps the image side of the cache small enough for the whole thing to work.\n\nOne thing the paper flags that is easy to miss: R-SWA is not OCR-specific. It is a general-purpose attention mechanism for any long-horizon transcription-shaped task, ASR included.\n\nYou need an NVIDIA GPU. There is no CPU or Apple Silicon path in the official repo.\n\nThe weights are bfloat16, so roughly 6 GB just to load the model, before activations and image tokens. A 12 GB card is a sane floor for single-image work; for multi-page runs at 32K context you will want more headroom. The maintainers tested on Python 3.12.3 with CUDA 12.9.\n\nIf you just want to see output before committing to any of this, there is a [Hugging Face Space](https://huggingface.co/spaces/baidu/Unlimited-OCR) you can drop a file into.\n\nThis is the fastest way to a working result. Set up the environment:\n\n```\npython -m venv .venv\nsource .venv/bin/activate\n\npip install torch==2.10.0 torchvision==0.25.0 transformers==4.57.1\npip install Pillow==12.1.1 matplotlib==3.10.8 einops==0.8.2\npip install addict==2.4.0 easydict==1.13 pymupdf==1.27.2.2 psutil==7.2.2\n```\n\nPin these versions. The model ships custom modeling code via `trust_remote_code`\n\n, and that code is written against these specific releases. Version drift here produces confusing import errors rather than clean failures.\n\nNow a single image:\n\n``` python\nimport torch\nfrom transformers import AutoModel, AutoTokenizer\n\nmodel_name = 'baidu/Unlimited-OCR'\n\ntokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)\nmodel = AutoModel.from_pretrained(\n    model_name,\n    trust_remote_code=True,\n    use_safetensors=True,\n    torch_dtype=torch.bfloat16,\n)\nmodel = model.eval().cuda()\n\nmodel.infer(\n    tokenizer,\n    prompt='<image>document parsing.',\n    image_file='your_image.jpg',\n    output_path='your/output/dir',\n    base_size=1024, image_size=640, crop_mode=True,   # gundam mode\n    max_length=32768,\n    no_repeat_ngram_size=35, ngram_window=128,\n    save_results=True,\n)\n```\n\nTwo details in there matter more than they look:\n\n**The prompt must literally start with <image>.** It is not decoration, it is the placeholder token the visual features get spliced into. Drop it and you will get output that looks like the model hallucinating a document it never saw.\n\n** no_repeat_ngram_size and ngram_window are load-bearing.** Long-horizon generation on repetitive layouts, think a table of contents or a price list, can send the model into a loop where it happily emits the same row forever. These parameters block any 35-gram from repeating inside a sliding window. Do not remove them because they look like tuning noise.\n\nSingle-image inference gives you two configurations, and the naming is not self-explanatory:\n\n| Mode | Settings | Use for |\n|---|---|---|\n| gundam | `base_size=1024, image_size=640, crop_mode=True` |\nSingle images, especially dense ones. Crops the image into tiles and processes each, so small text survives. |\n| base | `base_size=1024, image_size=1024, crop_mode=False` |\nWhole image at once. Required for multi-page. |\n\nMulti-page and PDF paths only support base mode. That is a hard constraint, not a default.\n\n```\nmodel.infer_multi(\n    tokenizer,\n    prompt='<image>Multi page parsing.',\n    image_files=['page1.png', 'page2.png', 'page3.png'],\n    output_path='your/output/dir',\n    image_size=1024,\n    max_length=32768,\n    no_repeat_ngram_size=35, ngram_window=1024,\n    save_results=True,\n)\n```\n\nNote that `ngram_window`\n\njumps from 128 to 1024 here. Across many pages there is legitimately more repeated structure, so the repeat-detection window has to widen to keep up.\n\nThere is no direct PDF entry point. You rasterize first, then feed the images to `infer_multi`\n\n:\n\n``` python\nimport os, tempfile\nimport fitz  # PyMuPDF\n\ndef pdf_to_images(pdf_path, dpi=300):\n    doc = fitz.open(pdf_path)\n    tmp_dir = tempfile.mkdtemp(prefix='pdf_ocr_')\n    mat = fitz.Matrix(dpi / 72, dpi / 72)\n    paths = []\n    for i, page in enumerate(doc):\n        out = os.path.join(tmp_dir, f'page_{i+1:04d}.png')\n        page.get_pixmap(matrix=mat).save(out)\n        paths.append(out)\n    doc.close()\n    return paths\n\nmodel.infer_multi(\n    tokenizer,\n    prompt='<image>Multi page parsing.',\n    image_files=pdf_to_images('your_doc.pdf', dpi=300),\n    output_path='your/output/dir',\n    image_size=1024,\n    max_length=32768,\n    no_repeat_ngram_size=35, ngram_window=1024,\n    save_results=True,\n)\n```\n\n300 DPI is the recommended default. Going lower to save memory costs you small text and table rules, which is usually the exact content you cared about.\n\nTransformers is fine for evaluating the model. For a service handling concurrent requests, run SGLang and talk to it over an OpenAI-compatible API.\n\nSet up the environment. Note that SGLang ships as a local wheel in the repo rather than from PyPI:\n\n```\nuv venv --python 3.12\nsource .venv/bin/activate\n\nuv pip install wheel/sglang-0.0.0.dev11416+g92e8bb79e-py3-none-any.whl\nuv pip install kernels==0.11.7\nuv pip install pymupdf==1.27.2.2\n```\n\nSmall heads-up: the README prose mentions pinning `kernels==0.9.0`\n\nwhile the command block right beneath it installs `0.11.7`\n\n. Follow the command block. If you hit kernel-related errors, that mismatch is the first thing to check against the current repo state.\n\nLaunch the server:\n\n```\npython -m sglang.launch_server \\\n    --model baidu/Unlimited-OCR \\\n    --served-model-name Unlimited-OCR \\\n    --attention-backend fa3 \\\n    --page-size 1 \\\n    --mem-fraction-static 0.8 \\\n    --context-length 32768 \\\n    --enable-custom-logit-processor \\\n    --disable-overlap-schedule \\\n    --skip-server-warmup \\\n    --host 0.0.0.0 \\\n    --port 10000\n```\n\nThe flags that are not optional:\n\n`--enable-custom-logit-processor`\n\n— the no-repeat-ngram processor runs as a custom logit processor. Without this flag, requests referencing it fail.`--attention-backend fa3`\n\n— FlashAttention 3, which requires Hopper-class hardware. On older GPUs you will need a different backend.`--page-size 1`\n\nand `--disable-overlap-schedule`\n\n— R-SWA's cache eviction does not play nicely with the usual paged-attention and overlapped-scheduling optimizations.Then a client. The image goes in as a base64 data URL, standard OpenAI vision format, with a couple of model-specific extras:\n\n``` python\nimport base64, json, os\nimport requests\nfrom sglang.srt.sampling.custom_logit_processor import (\n    DeepseekOCRNoRepeatNGramLogitProcessor,\n)\n\nserver_url = \"http://127.0.0.1:10000\"\nsession = requests.Session()\nsession.trust_env = False\n\ndef encode_image(image_path):\n    ext = os.path.splitext(image_path)[1].lower()\n    mime = \"image/jpeg\" if ext in (\".jpg\", \".jpeg\") else f\"image/{ext.lstrip('.')}\"\n    with open(image_path, \"rb\") as f:\n        data = base64.b64encode(f.read()).decode(\"utf-8\")\n    return {\"type\": \"image_url\", \"image_url\": {\"url\": f\"data:{mime};base64,{data}\"}}\n\ndef generate(prompt, image_paths, image_mode, ngram_window):\n    content = [{\"type\": \"text\", \"text\": prompt}] + [\n        encode_image(p) for p in image_paths\n    ]\n    payload = {\n        \"model\": \"Unlimited-OCR\",\n        \"messages\": [{\"role\": \"user\", \"content\": content}],\n        \"temperature\": 0,\n        \"skip_special_tokens\": False,\n        \"images_config\": {\"image_mode\": image_mode},\n        \"custom_logit_processor\": DeepseekOCRNoRepeatNGramLogitProcessor.to_str(),\n        \"custom_params\": {\"ngram_size\": 35, \"window_size\": ngram_window},\n        \"stream\": True,\n    }\n    response = session.post(\n        f\"{server_url}/v1/chat/completions\",\n        headers={\"Content-Type\": \"application/json\"},\n        data=json.dumps(payload),\n        timeout=1200,\n        stream=True,\n    )\n    response.raise_for_status()\n\n    chunks = []\n    for line in response.iter_lines(chunk_size=1, decode_unicode=True):\n        if not line or not line.startswith(\"data: \"):\n            continue\n        data = line[len(\"data: \"):]\n        if data == \"[DONE]\":\n            break\n        delta = json.loads(data)[\"choices\"][0].get(\"delta\", {}).get(\"content\", \"\")\n        if delta:\n            print(delta, end=\"\", flush=True)\n            chunks.append(delta)\n    return \"\".join(chunks)\n\ngenerate(\"document parsing.\", [\"your_image.jpg\"],\n         image_mode=\"gundam\", ngram_window=128)\n```\n\n`temperature: 0`\n\nand `skip_special_tokens: False`\n\nare both deliberate. This is transcription, not generation, so any sampling randomness is pure downside. And the special tokens carry layout structure you want in the output.\n\nThe 1200 second timeout is also not paranoia. Long documents take a while, which is precisely why you want streaming.\n\nThe repo ships `infer.py`\n\n, which starts the SGLang server for you and fires concurrent requests at it:\n\n```\n# A directory of images\npython infer.py \\\n    --image_dir ./examples/images \\\n    --output_dir ./outputs \\\n    --concurrency 8 \\\n    --image_mode gundam\n\n# A PDF\npython infer.py \\\n    --pdf ./examples/document.pdf \\\n    --output_dir ./outputs \\\n    --concurrency 8 \\\n    --image_mode gundam\n```\n\nUseful extras: `--model_dir`\n\naccepts a local path or a Hugging Face ID, `--gpu`\n\nsets `CUDA_VISIBLE_DEVICES`\n\n, and `--server_log`\n\nputs server output somewhere you can read it.\n\nStart concurrency low. Eight is the documented example, but the right number depends on your VRAM and how long your documents are.\n\nIf you want to skip environment management entirely:\n\n```\n# Default, CUDA 13.0\ndocker pull vllm/vllm-openai:unlimited-ocr\n\n# Hopper GPUs, CUDA 12.9\ndocker pull vllm/vllm-openai:unlimited-ocr-cu129\ndocker run --rm --gpus all --network host --ipc host \\\n    vllm/vllm-openai:unlimited-ocr \\\n    baidu/Unlimited-OCR \\\n    --trust-remote-code \\\n    --logits_processors vllm.model_executor.models.unlimited_ocr:NGramPerReqLogitsProcessor \\\n    --no-enable-prefix-caching \\\n    --mm-processor-cache-gb 0\n```\n\nSame story as SGLang regarding the ngram processor, it has to be registered at server start. Prefix caching and the multimodal processor cache are both disabled because R-SWA's fixed-window cache invalidates the assumptions those optimizations make.\n\nFull details are in the [official vLLM recipe](https://recipes.vllm.ai/baidu/Unlimited-OCR).\n\n| Parameter | Single image | Multi-page / PDF |\n|---|---|---|\n| mode | gundam or base | base only |\n`image_size` |\n640 (gundam) / 1024 (base) | 1024 |\n`crop_mode` |\nTrue (gundam) / False (base) | False |\n`ngram_window` |\n128 | 1024 |\n`no_repeat_ngram_size` |\n35 | 35 |\n`max_length` |\n32768 | 32768 |\n| prompt | `<image>document parsing.` |\n`<image>Multi page parsing.` |\n\nReach for something else if you need bounding boxes and per-word confidence scores, since this is an end-to-end model producing Markdown, not a detection-plus-recognition pipeline. Same if you are OCRing short receipts or single lines, where a 3B model on a GPU is enormous overkill compared to PaddleOCR or Tesseract. And if you have no NVIDIA GPU, there is no supported path today.\n\nThe interesting claim here is not the benchmark number. It is that a fixed-size KV cache made the model both faster and more accurate on long documents, when efficiency work almost always costs you quality somewhere.\n\nIf R-SWA generalizes the way the authors suggest, the same trick applies to any task where a model transcribes a long input into a long output while keeping the source in view. Long-form ASR is the obvious next one.", "url": "https://wpnews.pro/news/unlimited-ocr-parsing-a-40-page-pdf-in-one-pass-without-your-gpu-melting", "canonical_source": "https://dev.to/arshtechpro/unlimited-ocr-parsing-a-40-page-pdf-in-one-pass-without-your-gpu-melting-4mc4", "published_at": "2026-07-24 17:19:34+00:00", "updated_at": "2026-07-24 17:33:48.556078+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "computer-vision", "large-language-models", "ai-research"], "entities": ["Baidu", "Unlimited-OCR", "DeepSeek-OCR", "NVIDIA", "Hugging Face", "SGLang", "DeepEncoder", "R-SWA"], "alternates": {"html": "https://wpnews.pro/news/unlimited-ocr-parsing-a-40-page-pdf-in-one-pass-without-your-gpu-melting", "markdown": "https://wpnews.pro/news/unlimited-ocr-parsing-a-40-page-pdf-in-one-pass-without-your-gpu-melting.md", "text": "https://wpnews.pro/news/unlimited-ocr-parsing-a-40-page-pdf-in-one-pass-without-your-gpu-melting.txt", "jsonld": "https://wpnews.pro/news/unlimited-ocr-parsing-a-40-page-pdf-in-one-pass-without-your-gpu-melting.jsonld"}}