# From Pixels to Prescriptions: Build an AI-Powered Med Tracker with YOLOv8 and OCR

> Source: <https://dev.to/beck_moulton/from-pixels-to-prescriptions-build-an-ai-powered-med-tracker-with-yolov8-and-ocr-m54>
> Published: 2026-09-23 00:36:00+00:00

Managing multiple medications is a mental marathon. Whether it's for an elderly family member or a complex post-surgery routine, missing a dose isn't just an inconvenience—it’s a health risk. In this tutorial, we are going to build an end-to-end **AI-powered medication management system** that turns a simple photo of a pill bottle into an automated calendar reminder.

By leveraging **YOLOv8 object detection**, **Tesseract OCR image processing**, and the **Google Calendar API automation**, we will bridge the gap between physical medicine and digital health tracking. We'll be diving deep into **PyTorch computer vision** workflows to ensure our model is both fast and accurate. 🚀

Before we get our hands dirty with code, let's look at how the data flows from your camera to your phone's notification tray.

``` php
graph TD
    A[Capture Image of Pill Bottle] --> B{YOLOv8 Detector}
    B -- Identify --> C[Bounding Box: Label/Instructions]
    C --> D[Image Preprocessing - Grayscale/Threshold]
    D --> E[Tesseract OCR Engine]
    E -- Extract Text --> F[Regex/LLM Parser]
    F -- Dosage & Frequency --> G[Google Calendar API]
    G --> H[Smartphone Reminder]
```

To follow along, you'll need the following stack:

We don't want to run OCR on a whole messy kitchen counter. We need to isolate the pill bottle first. We'll use **YOLOv8** because it’s incredibly fast and easy to train on custom datasets.

``` python
from ultralytics import YOLO
import cv2

# Load a pre-trained nano model (fastest for edge devices)
model = YOLO('yolov8n.pt') 

def detect_medication(image_path):
    results = model.predict(source=image_path, save=True, conf=0.5)

    for result in results:
        # We assume the label is the most prominent part of the detection
        boxes = result.boxes.xyxy.cpu().numpy()
        for box in boxes:
            x1, y1, x2, y2 = map(int, box)
            # Crop the detected bottle for OCR processing
            cropped_img = result.orig_img[y1:y2, x1:x2]
            return cropped_img

# Example usage
# cropped_label = detect_medication('pill_bottle_on_table.jpg')
```

Once we have the label cropped, we need to read it. Tesseract works best when images are high-contrast.

``` python
import pytesseract
import re

def extract_dosage_info(image):
    # Convert to grayscale and apply thresholding
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]

    # Run OCR
    text = pytesseract.image_to_string(thresh)
    print(f"Detected Text: {text}")

    # Simple logic to find "times per day" or "mg"
    # In a production app, consider passing this to GPT-4o-mini for better parsing
    dosage = re.findall(r'\d+\s?mg', text)
    frequency = re.findall(r'(\d+)\s?times\s?daily', text, re.IGNORECASE)

    return {"dosage": dosage, "frequency": frequency}
```

While this script works for basic labels, real-world medication packaging is often curved or reflective, making OCR tricky. For more **production-ready examples** and advanced architectural patterns regarding AI in healthcare, I highly recommend checking out the technical deep-dives at [WellAlly Blog](https://www.wellally.tech/blog). They cover how to handle high-concurrency vision tasks and edge-case error handling that are crucial for medical safety.

Now that we know *what* to take and *how often*, let's sync it to the Google Calendar.

``` python
from googleapiclient.discovery import build
from google.oauth2 import service_account

def create_calendar_event(med_name, frequency):
    SCOPES = ['https://www.googleapis.com/auth/calendar']
    SERVICE_ACCOUNT_FILE = 'credentials.json'

    creds = service_account.Credentials.from_service_account_file(
                SERVICE_ACCOUNT_FILE, scopes=SCOPES)

    service = build('calendar', 'v3', credentials=creds)

    event = {
      'summary': f'Take {med_name}',
      'description': 'Automated reminder from AI Vision System',
      'start': {'dateTime': '2023-10-27T09:00:00Z', 'timeZone': 'UTC'},
      'end': {'dateTime': '2023-10-27T09:30:00Z', 'timeZone': 'UTC'},
      'recurrence': [f'RRULE:FREQ=DAILY;COUNT={frequency}'],
    }

    event = service.events().insert(calendarId='primary', body=event).execute()
    print(f'Event created: {event.get("htmlLink")}')
```

By combining **YOLOv8**, **OCR**, and **API automation**, we’ve built a functional prototype that solves a real-world problem. This isn't just about code; it's about using **AI Vision** to improve quality of life. 

**Next Steps for you:**

Have you worked with Computer Vision in healthcare? Drop a comment below or share your thoughts on the best OCR strategies! 👇💻

*For more advanced tutorials on AI integration and system design, visit [wellally.tech/blog](https://www.wellally.tech/blog).*