Managing multiple medications is a mental marathon. Whether it's for an elderly family member or a complex post-surgery routine, missing a dose isn't just an inconvenience—it’s a health risk. In this tutorial, we are going to build an end-to-end AI-powered medication management system that turns a simple photo of a pill bottle into an automated calendar reminder.
By leveraging YOLOv8 object detection, Tesseract OCR image processing, and the Google Calendar API automation, we will bridge the gap between physical medicine and digital health tracking. We'll be diving deep into PyTorch computer vision workflows to ensure our model is both fast and accurate. 🚀
Before we get our hands dirty with code, let's look at how the data flows from your camera to your phone's notification tray.
graph TD
A[Capture Image of Pill Bottle] --> B{YOLOv8 Detector}
B -- Identify --> C[Bounding Box: Label/Instructions]
C --> D[Image Preprocessing - Grayscale/Threshold]
D --> E[Tesseract OCR Engine]
E -- Extract Text --> F[Regex/LLM Parser]
F -- Dosage & Frequency --> G[Google Calendar API]
G --> H[Smartphone Reminder]
To follow along, you'll need the following stack:
We don't want to run OCR on a whole messy kitchen counter. We need to isolate the pill bottle first. We'll use YOLOv8 because it’s incredibly fast and easy to train on custom datasets.
from ultralytics import YOLO
import cv2
model = YOLO('yolov8n.pt')
def detect_medication(image_path):
results = model.predict(source=image_path, save=True, conf=0.5)
for result in results:
boxes = result.boxes.xyxy.cpu().numpy()
for box in boxes:
x1, y1, x2, y2 = map(int, box)
cropped_img = result.orig_img[y1:y2, x1:x2]
return cropped_img
Once we have the label cropped, we need to read it. Tesseract works best when images are high-contrast.
import pytesseract
import re
def extract_dosage_info(image):
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
text = pytesseract.image_to_string(thresh)
print(f"Detected Text: {text}")
dosage = re.findall(r'\d+\s?mg', text)
frequency = re.findall(r'(\d+)\s?times\s?daily', text, re.IGNORECASE)
return {"dosage": dosage, "frequency": frequency}
While this script works for basic labels, real-world medication packaging is often curved or reflective, making OCR tricky. For more production-ready examples and advanced architectural patterns regarding AI in healthcare, I highly recommend checking out the technical deep-dives at WellAlly Blog. They cover how to handle high-concurrency vision tasks and edge-case error handling that are crucial for medical safety.
Now that we know what to take and how often, let's sync it to the Google Calendar.
from googleapiclient.discovery import build
from google.oauth2 import service_account
def create_calendar_event(med_name, frequency):
SCOPES = ['https://www.googleapis.com/auth/calendar']
SERVICE_ACCOUNT_FILE = 'credentials.json'
creds = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
service = build('calendar', 'v3', credentials=creds)
event = {
'summary': f'Take {med_name}',
'description': 'Automated reminder from AI Vision System',
'start': {'dateTime': '2023-10-27T09:00:00Z', 'timeZone': 'UTC'},
'end': {'dateTime': '2023-10-27T09:30:00Z', 'timeZone': 'UTC'},
'recurrence': [f'RRULE:FREQ=DAILY;COUNT={frequency}'],
}
event = service.events().insert(calendarId='primary', body=event).execute()
print(f'Event created: {event.get("htmlLink")}')
By combining YOLOv8, OCR, and API automation, we’ve built a functional prototype that solves a real-world problem. This isn't just about code; it's about using AI Vision to improve quality of life.
Next Steps for you:
Have you worked with Computer Vision in healthcare? Drop a comment below or share your thoughts on the best OCR strategies! 👇💻
For more advanced tutorials on AI integration and system design, visit wellally.tech/blog.