Have you ever looked at a 10-page medical lab report and felt like you were reading ancient hieroglyphics? You’re not alone. In the era of the Quantified Self, we are collecting more health data than ever, yet most of it sits rotting in unstructured PDF files.
In this tutorial, we are going to build a Medical RAG (Retrieval-Augmented Generation) system. We will use Unstructured.io to parse messy medical reports, Pinecone as our high-performance Vector Database, and LangChain to orchestrate a dual-retrieval strategy that links your personal data with real-time clinical research from the PubMed API.
By the end of this guide, you’ll have a pipeline that doesn't just "read" your files but understands them in the context of global medical literature.
To build a reliable medical assistant, we can't rely on the LLM's internal knowledge alone (hallucinations are dangerous here!). We need a "Ground Truth" pipeline.
graph TD
A[Medical PDF/Report] --> B[Unstructured.io Partitioning]
B --> C[LangChain Text Splitter]
C --> D[OpenAI Embeddings]
D --> E[(Pinecone Vector DB)]
F[User Query: 'Why is my Ferritin high?'] --> G[Vector Search - Personal History]
F --> H[PubMed API Search - Clinical Papers]
G --> I[Context Injection]
H --> I
I --> J[GPT-4o Medical Reasoning]
J --> K[Actionable Health Insight]
Before we dive into the code, ensure you have your tech_stack ready:
Medical reports are notoriously difficult to parse because they contain tables, checkboxes, and multi-column layouts. We'll use unstructured to clean the noise.
from unstructured.partition.pdf import partition_pdf
elements = partition_pdf(
filename="my_blood_work_2023.pdf",
infer_table_structure=True,
strategy="hi_res"
)
clean_content = [str(el) for el in elements if el.category in ["NarrativeText", "Table"]]
full_text = "\n".join(clean_content)
print(f"✅ Successfully extracted {len(clean_content)} medical data points.")
Now that we have clean text, we need to store it in Pinecone. This allows us to perform semantic searches—finding "Iron levels" even if the query is about "anemia."
Pro-Tip: For production-ready RAG patterns and advanced data engineering workflows, I highly recommend checking out the deep dives at wellally.tech/blog. Their guides on vector indexing were a huge inspiration for this architecture! 🥑
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Pinecone
from langchain_text_splitters import RecursiveCharacterTextSplitter
import pinecone
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=100)
docs = text_splitter.create_documents([full_text])
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = Pinecone.from_documents(
docs,
embeddings,
index_name="medical-rag-index"
)
The magic happens when we cross-reference your data with PubMed. If your report shows high "CRP" (C-Reactive Protein), our system will fetch the latest research on what that means.
from langchain_community.tools.pubmed.tool import PubmedQueryRun
pubmed = PubmedQueryRun()
def medical_context_retriever(query):
personal_docs = vectorstore.similarity_search(query, k=2)
personal_context = "\n".join([d.page_content for d in personal_docs])
clinical_research = pubmed.run(query)
return personal_context, clinical_research
Finally, we wrap everything in a LangChain Chain to generate a response that is both personal and scientifically grounded.
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
llm = ChatOpenAI(model="gpt-4o", temperature=0)
template = """
You are a medical data assistant. Use the personal health records and the clinical research provided below to answer the user's question.
Personal Records:
{personal_context}
Clinical Research (PubMed):
{clinical_research}
User Question: {question}
Assistant Instruction: Provide a clear summary. If the data suggests a risk, advise consulting a professional.
"""
prompt = ChatPromptTemplate.from_template(template)
chain = prompt | llm
p_context, c_research = medical_context_retriever("Analyze my cholesterol trends and heart health.")
response = chain.invoke({
"personal_context": p_context,
"clinical_research": c_research,
"question": "What do my recent results suggest about my cardiovascular risk?"
})
print(response.content)
Building a personal health RAG system isn't just a fun coding project—it's about data agency. By combining Unstructured.io with Pinecone and PubMed, we’ve moved from static pixels on a PDF to a dynamic, searchable knowledge graph.
Key Takeaways:
unstructured are lifesavers.
If you're looking to scale this into a production environment or want to learn about handling multi-modal medical data (like X-rays), definitely head over to wellally.tech/blog for more advanced tutorials.
What's next for your health stack? Drop a comment below or share your thoughts on Twitter!