{"slug": "etl-for-longevity-automating-blood-test-analysis-with-gpt-4-and-instructor", "title": "ETL for Longevity: Automating Blood Test Analysis with GPT-4 and Instructor", "summary": "A developer detailed a method for automating blood test analysis using GPT-4 and the Instructor library, converting unstructured PDF reports into structured data stored in PostgreSQL. The pipeline leverages Pydantic schemas to ensure accurate extraction and enables time-series health tracking.", "body_md": "We’ve all been there: you get your annual blood work done, and the hospital sends you a cryptic 5-page PDF. You want to track your glucose or cholesterol levels over time, but the data is trapped in a non-standardized table format. Doing this manually is a nightmare.\n\nIn the world of **Data Engineering**, this is a classic \"unstructured to structured\" problem. Today, we are building a robust **LLM-powered ETL pipeline** to automate the ingestion of blood test reports into a **PostgreSQL** database. By leveraging **Structured Outputs** with the `Instructor`\n\nlibrary, we can turn messy PDFs into clean, queryable health insights.\n\nWhether you are building a personal longevity dashboard or a professional health informatics tool, mastering this **structured data extraction** workflow is a game-changer for your **health data pipeline**.\n\nThe flow is straightforward but powerful. We extract the raw text, pass it to a Large Language Model (LLM) constrained by a strict Pydantic schema, and then commit that validated data to our database.\n\n``` php\ngraph TD\n    A[Blood Test PDF] -->|PDFPlumber| B(Raw Text Extraction)\n    B -->|Text Prompt| C{Instructor + GPT-4o}\n    C -->|Validated JSON| D[Pydantic Model]\n    D -->|SQL Insert| E[(PostgreSQL Database)]\n    E -->|Query| F[Streamlit Dashboard]\n\n    subgraph \"Validation Layer\"\n    C\n    D\n    end\n```\n\nTo follow along, you'll need:\n\nThe secret sauce to a reliable ETL is a strict schema. We use `Pydantic`\n\nto define exactly what a \"Blood Test\" looks like. This ensures the LLM doesn't hallucinate random units or field names.\n\n``` python\nfrom pydantic import BaseModel, Field\nfrom typing import List, Optional\nfrom datetime import date\n\nclass BloodMarker(BaseModel):\n    name: str = Field(..., description=\"Common name of the marker, e.g., LDL Cholesterol\")\n    value: float = Field(..., description=\"The numerical value recorded\")\n    unit: str = Field(..., description=\"The unit of measurement, e.g., mg/dL\")\n    reference_range: Optional[str] = Field(None, description=\"The normal range provided by the lab\")\n\nclass BloodReport(BaseModel):\n    report_date: date\n    hospital_name: str\n    patient_name: str\n    markers: List[BloodMarker]\n```\n\nHospital PDFs are notoriously inconsistent. Some use tables, others use key-value columns. We use `PDFPlumber`\n\nto get the raw text and let `Instructor`\n\nhandle the heavy lifting of understanding the context.\n\n``` python\nimport pdfplumber\nimport instructor\nfrom openai import OpenAI\n\n# Initialize Instructor-patched client\nclient = instructor.from_openai(OpenAI())\n\ndef extract_blood_data(pdf_path: str) -> BloodReport:\n    # 1. Extract raw text from PDF\n    with pdfplumber.open(pdf_path) as pdf:\n        raw_text = \"\\n\".join([page.extract_text() for page in pdf.pages])\n\n    # 2. Use Instructor to parse the text into our Pydantic model\n    report = client.chat.completions.create(\n        model=\"gpt-4o\",\n        response_model=BloodReport,\n        messages=[\n            {\"role\": \"system\", \"content\": \"You are a specialized medical data extractor.\"},\n            {\"role\": \"user\", \"content\": f\"Extract all blood markers from this text: {raw_text}\"}\n        ],\n    )\n    return report\n\n# Usage\n# data = extract_blood_data(\"my_blood_work_2023.pdf\")\n# print(data.model_dump_json(indent=2))\n```\n\nOnce we have a validated `BloodReport`\n\nobject, saving it to a relational database allows us to perform time-series analysis (e.g., \"Show me my Vitamin D levels over the last 3 years\").\n\n``` python\nimport psycopg2\nfrom psycopg2.extras import execute_values\n\ndef save_to_db(report: BloodReport):\n    conn = psycopg2.connect(\"dbname=longevity_db user=postgres password=secret\")\n    cur = conn.cursor()\n\n    # Simple logic to insert markers\n    query = \"\"\"\n    INSERT INTO blood_results (report_date, marker_name, value, unit)\n    VALUES %s\n    \"\"\"\n    data_points = [\n        (report.report_date, m.name, m.value, m.unit) \n        for m in report.markers\n    ]\n\n    execute_values(cur, query, data_points)\n    conn.commit()\n    cur.close()\n    conn.close()\n    print(\"🚀 Data successfully ingested!\")\n```\n\nWhile this script works for individual files, production-grade longevity apps require handling multi-page tables, OCR for scanned images, and data normalization (e.g., converting 'mg/dL' to 'mmol/L').\n\nFor a deeper dive into production-ready AI architectures and advanced data engineering patterns for health tech, I highly recommend checking out the ** WellAlly Tech Blog**. They cover extensively how to scale these LLM workflows and handle complex medical data privacy requirements.\n\nFinally, let's wrap this in a simple UI. Streamlit allows us to upload a PDF and immediately see our historical trends.\n\n``` python\nimport streamlit as st\n\nst.title(\"Longevity Tracker 🧬\")\nuploaded_file = st.file_uploader(\"Upload Blood Test PDF\", type=\"pdf\")\n\nif uploaded_file:\n    with st.spinner(\"Processing medical data...\"):\n        # Logic to extract and save...\n        st.success(\"Analysis Complete!\")\n        # Use st.line_chart to show historical data from SQL\n```\n\nBy combining **Instructor** for structured LLM outputs and **PostgreSQL** for long-term storage, we've turned a manual, error-prone task into a seamless automated pipeline. This is the foundation of \"Learning in Public\"—taking a personal pain point (messy medical PDFs) and solving it with a modern tech stack.\n\n**Ready to take your data engineering to the next level?**", "url": "https://wpnews.pro/news/etl-for-longevity-automating-blood-test-analysis-with-gpt-4-and-instructor", "canonical_source": "https://dev.to/beck_moulton/etl-for-longevity-automating-blood-test-analysis-with-gpt-4-and-instructor-37eh", "published_at": "2026-08-28 00:36:00+00:00", "updated_at": "2026-08-28 01:18:59.582729+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "artificial-intelligence"], "entities": ["GPT-4", "Instructor", "PostgreSQL", "PDFPlumber", "Pydantic", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/etl-for-longevity-automating-blood-test-analysis-with-gpt-4-and-instructor", "markdown": "https://wpnews.pro/news/etl-for-longevity-automating-blood-test-analysis-with-gpt-4-and-instructor.md", "text": "https://wpnews.pro/news/etl-for-longevity-automating-blood-test-analysis-with-gpt-4-and-instructor.txt", "jsonld": "https://wpnews.pro/news/etl-for-longevity-automating-blood-test-analysis-with-gpt-4-and-instructor.jsonld"}}