{"slug": "a-deep-dive-into-the-microsoft-foundry-document-intelligence-sdk-from-pdf-to", "title": "A Deep Dive into the Microsoft Foundry Document Intelligence SDK: From PDF to Structured Data", "summary": "A developer published a hands-on deep dive into Microsoft's Document Intelligence SDK (formerly Form Recognizer), demonstrating how to convert scanned invoices, contracts, and receipts into structured data for RAG pipelines. The walkthrough covers using the DocumentIntelligenceClient and DocumentIntelligenceAdministrationClient with prebuilt models like prebuilt-layout, prebuilt-invoice, and prebuilt-receipt, extracting layout as GitHub-flavored markdown, pulling named fields with confidence scores, classifying documents, and training custom extraction models on labeled data.", "body_md": "Most \"RAG over PDFs\" pipelines have a step nobody talks about much: something has to turn a scanned invoice, a multi-column contract, or a photographed receipt into text a model can actually reason over. On Microsoft's stack, that something is usually the Document Intelligence SDK, formerly Form Recognizer, and it's worth understanding on its own terms rather than treating it as a black box that happens before the interesting part starts.\n\nThis is a hands-on deep dive into that SDK specifically. Not a tour of every Foundry Tools SDK, Vision and Speech and Content Safety each deserve their own treatment, but a real build using Document Intelligence: extracting layout as clean markdown, pulling structured fields out of a known document type, classifying documents before routing them, and training a custom extraction model on your own labeled data.\n\nTwo clients, and three kinds of model, cover almost everything this SDK does:\n\n`DocumentIntelligenceClient`` begin_analyze_document`, and a `model_id` parameter decides what kind of analysis happens. It's a long-running operation, so every call returns a poller.`DocumentIntelligenceAdministrationClient`` prebuilt-layout`, `prebuilt-invoice`, `prebuilt-receipt`, `prebuilt-idDocument`, `prebuilt-read`, and others) handle common, well-known document shapes out of the box. No training required.\n\n```\npip install azure-ai-documentintelligence azure-identity\npython\nfrom azure.ai.documentintelligence import DocumentIntelligenceClient\nfrom azure.core.credentials import AzureKeyCredential\n\nendpoint = \"https://YOUR-RESOURCE.cognitiveservices.azure.com\"\nclient = DocumentIntelligenceClient(endpoint=endpoint, credential=AzureKeyCredential(\"YOUR-KEY\"))\n```\n\nFor anything past local experimentation, swap the key for `DefaultAzureCredential` and an RBAC role scoped to the resource, the same pattern every other Foundry-adjacent SDK in this series has used.\n\nThis is the single most useful call in the whole SDK if your end goal is feeding documents into a RAG pipeline. `prebuilt-layout` doesn't just extract text, it understands headings, tables, and section structure, and it can hand all of that back as GitHub-flavored markdown instead of a flat text blob.\n\n``` python\nfrom azure.ai.documentintelligence.models import AnalyzeDocumentRequest, DocumentContentFormat\n\nwith open(\"contract.pdf\", \"rb\") as f:\n    poller = client.begin_analyze_document(\n        \"prebuilt-layout\",\n        AnalyzeDocumentRequest(bytes_source=f.read()),\n        output_content_format=DocumentContentFormat.MARKDOWN,\n    )\nresult = poller.result()\nprint(result.content[:500])\n```\n\n`result.content` is now a markdown string, headings as `#`, tables as GFM pipe tables, page structure preserved. That matters more than it sounds like it should: a table flattened into plain text loses its row and column relationships, and a model reasoning over that text has to reconstruct structure it was never actually given. Markdown output keeps the structure intact.\n\nFor document types Document Intelligence already knows, invoices are the clearest example, you get named fields back with a confidence score per field, not just raw text.\n\n```\nwith open(\"invoice.pdf\", \"rb\") as f:\n    poller = client.begin_analyze_document(\"prebuilt-invoice\", AnalyzeDocumentRequest(bytes_source=f.read()))\nresult = poller.result()\n\nfor doc in result.documents:\n    vendor = doc.fields.get(\"VendorName\")\n    total = doc.fields.get(\"InvoiceTotal\")\n    if vendor:\n        print(f\"Vendor: {vendor.value_string} (confidence: {vendor.confidence:.2f})\")\n    if total:\n        print(f\"Total: {total.value_currency.amount} (confidence: {total.confidence:.2f})\")\n```\n\nThat confidence score isn't decoration. It's the field you should actually branch on in production code, more on that in the production section below.\n\nA few optional capabilities aren't on by default, since they add processing cost, but are worth turning on deliberately rather than discovering you needed them after the fact:\n\n``` python\nfrom azure.ai.documentintelligence.models import AnalyzeDocumentRequest, DocumentAnalysisFeature\n\nwith open(\"shipping-label.pdf\", \"rb\") as f:\n    poller = client.begin_analyze_document(\n        \"prebuilt-layout\",\n        AnalyzeDocumentRequest(bytes_source=f.read()),\n        features=[DocumentAnalysisFeature.BARCODES, DocumentAnalysisFeature.FORMULAS],\n    )\n```\n\n`BARCODES` extracts barcode and QR code payloads directly, useful for shipping labels and inventory documents where the barcode carries the actual identifier the text doesn't repeat. `FORMULAS` pulls out mathematical expressions as LaTeX, relevant if you're processing scientific or financial documents where a formula matters more than the surrounding prose. There's also a high-resolution mode for documents where small print matters, at the cost of slower processing.\n\nReal intake pipelines rarely receive one document type. A classifier solves the \"what am I even looking at\" problem before you commit to an extraction model.\n\n``` python\nfrom azure.ai.documentintelligence import DocumentIntelligenceAdministrationClient\nfrom azure.ai.documentintelligence.models import (\n    BuildDocumentClassifierRequest,\n    ClassifierDocumentTypeDetails,\n    AzureBlobContentSource,\n)\n\nadmin_client = DocumentIntelligenceAdministrationClient(endpoint=endpoint, credential=AzureKeyCredential(\"YOUR-KEY\"))\n\npoller = admin_client.begin_build_classifier(\n    BuildDocumentClassifierRequest(\n        classifier_id=\"support-doc-classifier\",\n        doc_types={\n            \"invoice\": ClassifierDocumentTypeDetails(\n                azure_blob_source=AzureBlobContentSource(container_url=\"<SAS-url-to-invoices-container>\")\n            ),\n            \"contract\": ClassifierDocumentTypeDetails(\n                azure_blob_source=AzureBlobContentSource(container_url=\"<SAS-url-to-contracts-container>\")\n            ),\n        },\n    )\n)\nclassifier = poller.result()\n```\n\nYou need at least five sample documents per category to train a classifier at all, and more than that for anything you'd trust in production. Once it's built, classifying an incoming document is a single call:\n\n```\nwith open(\"unknown.pdf\", \"rb\") as f:\n    poller = client.begin_classify_document(\"support-doc-classifier\", AnalyzeDocumentRequest(bytes_source=f.read()))\nresult = poller.result()\nfor doc in result.documents:\n    print(f\"Classified as: {doc.doc_type} (confidence: {doc.confidence:.2f})\")\n```\n\nWhen a document type isn't invoices, receipts, or any of the other prebuilt shapes, train your own. This needs a set of labeled training documents in Blob Storage, produced through the labeling tool in Foundry's document intelligence studio or programmatically.\n\n```\nfrom azure.ai.documentintelligence.models import (\n    BuildDocumentModelRequest,\n    AzureBlobContentSource,\n    DocumentBuildMode,\n)\n\npoller = admin_client.begin_build_document_model(\n    BuildDocumentModelRequest(\n        model_id=\"acme-service-agreement-v1\",\n        build_mode=DocumentBuildMode.TEMPLATE,\n        azure_blob_source=AzureBlobContentSource(container_url=\"<SAS-url-to-training-container>\"),\n        description=\"Extraction model for Acme's standard service agreement template.\",\n    )\n)\nmodel = poller.result()\n```\n\nTwo build modes matter here, and they're not interchangeable. `TEMPLATE` mode is faster to train and works well when your documents follow a consistent visual layout, the same form filled out differently each time. `NEURAL` mode handles structural variation better, different layouts that still represent the same document type, at the cost of needing more training examples and longer build time. Start with `TEMPLATE` unless your documents genuinely vary in structure, not just content.\n\nOne naming constraint worth knowing before you hit it: a custom model ID can't start with `prebuilt-`, since that prefix is reserved for Microsoft's own models across every resource.\n\nThis is the detail that trips people up once they've also worked with the Foundry SDK or Agent Framework elsewhere in this series: Document Intelligence doesn't go through your Foundry project endpoint at all. It has its own resource, its own endpoint (`resource.cognitiveservices.azure.com`), and its own authentication scope. That's what \"Foundry Tools SDK\" actually means as a category, prebuilt AI services with tool-specific endpoints, distinct from the Foundry SDK's unified project endpoint that Agent Framework and the Responses API build on.\n\nThe practical upshot is the pipeline most teams actually want: run `prebuilt-layout` over incoming documents, get markdown back, and hand that markdown to a Foundry IQ Knowledge Base as a File Knowledge Source. Document Intelligence handles turning the PDF into clean, structured text. Foundry IQ handles chunking, embedding, and retrieval on top of it. Neither service needs to know the other exists, they just happen to compose well because markdown is a reasonable interchange format for both.\n\n`TEMPLATE` vs `NEURAL` is a real tradeoff, not a default to leave unexamined.\nThe Document Intelligence SDK is easy to undersell because the interesting part of most AI applications feels like it's happening somewhere else, in the model, in the retrieval layer, in the agent's reasoning. But the quality ceiling of everything downstream is set right here, at the point where a physical or scanned document either does or doesn't become text a model can actually use well. Layout extraction to markdown, confidence-aware field extraction, classifiers for mixed intake, and custom models for your own document shapes cover the large majority of real document-processing needs, and all four are a few lines of SDK code once you know which one you need. The judgment call was never really about the API. It's about matching the right one of these four tools to what's actually in your inbound documents.", "url": "https://wpnews.pro/news/a-deep-dive-into-the-microsoft-foundry-document-intelligence-sdk-from-pdf-to", "canonical_source": "https://dev.to/jubinsoni/a-deep-dive-into-the-microsoft-foundry-document-intelligence-sdk-from-pdf-to-structured-data-1mc8", "published_at": "2026-09-15 18:21:01+00:00", "updated_at": "2026-09-15 18:49:21.233052+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-products", "natural-language-processing", "ai-infrastructure"], "entities": ["Microsoft", "Document Intelligence SDK", "Form Recognizer", "DocumentIntelligenceClient", "DocumentIntelligenceAdministrationClient", "Azure", "DefaultAzureCredential", "Foundry Tools"], "alternates": {"html": "https://wpnews.pro/news/a-deep-dive-into-the-microsoft-foundry-document-intelligence-sdk-from-pdf-to", "markdown": "https://wpnews.pro/news/a-deep-dive-into-the-microsoft-foundry-document-intelligence-sdk-from-pdf-to.md", "text": "https://wpnews.pro/news/a-deep-dive-into-the-microsoft-foundry-document-intelligence-sdk-from-pdf-to.txt", "jsonld": "https://wpnews.pro/news/a-deep-dive-into-the-microsoft-foundry-document-intelligence-sdk-from-pdf-to.jsonld"}}