From Scribbles to Standards: Converting Unstructured Medical Notes to HL7 FHIR with GPT-4o A developer outlined a pipeline that uses GPT-4o with few-shot prompting and OpenAI's Structured Outputs to convert unstructured clinical notes into validated HL7 FHIR R4 Observation resources. The approach pairs Pydantic schema definitions with the API's response_format parameter to force model output into strict FHIR-compliant JSON, with invalid results routed to retry or human review. Let’s be honest: doctors are famous for two things—saving lives and having handwriting that looks like a caffeinated spider crawled across a page. For Data Engineers in the healthcare space, this "unstructured data" is a nightmare. Traditional OCR and Regex-based parsers often fail because medical notes aren't just messy; they are context-heavy. To achieve true healthcare interoperability , we need to transform these scribbles into the HL7 FHIR Fast Healthcare Interoperability Resources standard. In this tutorial, we are diving into the "Dark Arts" of medical data cleaning. We'll leverage GPT-4o , Few-shot prompting , and JSON Schema to build a robust pipeline that turns raw clinical text into valid FHIR resources. Before we write any code, we need a solid strategy. We aren't just "asking" the AI to format text; we are building a validation engine. php graph TD A Raw Unstructured Medical Note -- B{GPT-4o Engine} B -- |Few-Shot Context| C Extraction Logic C -- D JSON Schema Validation D -- |Valid| E FHIR Patient/Observation Resource D -- |Invalid| F Retry/Human-in-the-loop E -- G Clinical Data Repository style B fill: f9f,stroke: 333,stroke-width:2px style E fill: 00ff00,stroke: 333 To follow along, you'll need: openai , pydantic for schema validation , and a basic understanding of FHIR is strict. If your JSON doesn't match the resource definition exactly, the receiving server will reject it. We’ll use Pydantic to define a simplified FHIR Observation resource to ensure our LLM output is production-ready. python from pydantic import BaseModel, Field from typing import List, Optional class FHIRCoding BaseModel : system: str code: str display: str class FHIRQuantity BaseModel : value: float unit: str system: str = "http://unitsofmeasure.org" code: str class FHIRObservation BaseModel : resourceType: str = "Observation" status: str category: List dict code: dict Should contain a LOINC code subject: dict Reference to Patient valueQuantity: Optional FHIRQuantity = None note: Optional List dict = None The secret to converting "Dark Data" is Few-shot prompting . By providing the LLM with 2-3 examples of "Messy Input - Valid FHIR," we drastically reduce hallucinations. SYSTEM PROMPT = """ You are a specialized Medical Informatics Agent. Your task is to parse unstructured clinical notes into valid HL7 FHIR R4 JSON. Focus on extracting Vital Signs into 'Observation' resources. Use LOINC codes for observations e.g., 85354-9 for Blood Pressure . """ FEW SHOT EXAMPLES = """ Input: "Pt seen today. BP 120/80. HR 72. Complaining of mild headache." Output: { "resourceType": "Observation", "status": "final", "code": { "coding": {"system": "http://loinc.org", "code": "85354-9", "display": "Blood Pressure"} }, "component": {"code": {"text": "Systolic"}, "valueQuantity": {"value": 120, "unit": "mmHg"}}, {"code": {"text": "Diastolic"}, "valueQuantity": {"value": 80, "unit": "mmHg"}} } """ OpenAI recently released Structured Outputs , which forces the model to adhere strictly to a provided JSON Schema. This is a game-changer for Data Engineering pipelines. python import openai import json client = openai.OpenAI def extract medical data raw text: str : response = client.beta.chat.completions.parse model="gpt-4o-2024-08-06", messages= {"role": "system", "content": SYSTEM PROMPT}, {"role": "user", "content": f"Extract from this note: {raw text}"} , response format=FHIRObservation, Enforcing the Pydantic schema return response.choices 0 .message.parsed Example Usage raw note = "Subject: John Doe. The patient presents with a fever of 102F and a rapid heart rate of 110bpm." fhir data = extract medical data raw note print fhir data.model dump json indent=2 In the world of Medical Data Engineering , accuracy isn't just a metric; it's a safety requirement. Using LLMs as a "reasoning bridge" between raw text and structured standards like FHIR allows us to unlock insights from millions of legacy records that were previously "dark" unsearchable and unusable . 💡 Pro-Tip : For production-ready examples and advanced patterns on handling HIPAA-compliant data pipelines, check out the specialized guides over at the WellAlly Blog https://www.wellally.tech/blog . They have fantastic resources on scaling medical AI without compromising on data integrity. We’ve moved past the era of manual data entry. By combining the reasoning power of GPT-4o with the structural rigor of FHIR and Pydantic , we can automate the most tedious parts of healthcare tech. What's next? Are you working with medical data? What's the craziest "doctor handwriting" you've ever had to parse? Let me know in the comments 👇