Tired of Refreshing? Building a Smart Medical Appointment Agent with Playwright and LLM Function Calling 🏥🤖

We’ve all been there: waking up at 6:00 AM, frantically refreshing a hospital’s booking page, only to find that the "Expert Specialist" slots vanished in milliseconds. Traditional browser automation often fails here because hospital portals are notoriously clunky, filled with dynamic pop-ups, and inconsistent UI layouts.

In this guide, we are going to build a next-generation AI Agent for intelligent task automation. By combining the raw power of Playwright Python with LLM Function Calling, we’ll create a system that doesn't just "click" but "understands" the appointment flow. This approach moves us from brittle CSS selectors to resilient, reasoning-based automation.

Unlike traditional scripts that break when a button's ID changes from submit-01

to btn-confirm

, our agent uses a ReAct (Reason + Act) pattern. It observes the page, sends the simplified HTML/Accessibility tree to the LLM, and decides which tool to use next.

graph TD
    A[User Request: Book Dr. Smith] --> B{Agent Controller}
    B --> C[LLM Reasoning Engine]
    C -->|Decides Tool| D[Playwright Executor]
    D -->|Interaction| E[Hospital Portal]
    E -->|Page Content/Screenshot| D
    D -->|Observation| B
    B -->|State Storage| F[(Redis)]
    C -->|Final Confirmation| G[User Notified]

To follow this advanced tutorial, you'll need:

We need to give our LLM "hands." We do this by defining functions that Playwright will execute. The LLM won't write code; it will output JSON matching our function signatures.

import asyncio
from playwright.async_api import async_playwright

class MedicalAgentTools:
    def __init__(self, page):
        self.page = page

    async def navigate_to_department(self, dept_name: str):
        """Navigates to a specific hospital department link."""
        links = await self.page.get_by_role("link").all()
        for link in links:
            text = await link.inner_text()
            if dept_name in text:
                await link.click()
                return f"Successfully navigated to {dept_name}"
        return "Department not found."

    async def fill_patient_info(self, name: str, id_number: str):
        """Fills the appointment form with patient details."""
        await self.page.fill('input[placeholder="Patient Name"]', name)
        await self.page.fill('input[name="id_card"]', id_number)
        return "Patient info filled successfully."

The core logic involves sending the current page "state" (a simplified DOM) to the LLM. We use LLM Function Calling to let the model choose between navigating, filling forms, or solving a CAPTCHA.

import openai

async def run_agent_step(agent_tools, user_prompt):
    messages = [
        {"role": "system", "content": "You are a medical booking assistant. Use tools to complete the user's request."},
        {"role": "user", "content": user_prompt}
    ]

    tools = [
        {
            "type": "function",
            "function": {
                "name": "navigate_to_department",
                "parameters": {
                    "type": "object",
                    "properties": {"dept_name": {"type": "string"}}
                }
            }
        }
    ]

    response = await openai.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools
    )

In high-concurrency scenarios (like when a doctor’s schedule opens), you don't want to spawn 1000 browsers. We use Redis to cache session cookies and manage a queue of booking requests to prevent IP bans.

import redis

r = redis.Redis(host='localhost', port=6379, db=0)

async def save_session(context):
    state = await context.storage_state()
    r.set("medical_session_user_1", str(state))

async def load_session(context):
    state = r.get("medical_session_user_1")
    if state:
        await context.add_cookies(eval(state)['cookies'])

While this script works for personal use, scaling AI-driven automation for enterprise healthcare or high-traffic systems requires robust error handling, proxy rotation, and advanced DOM parsing.

For more production-ready patterns and deep dives into AI Agent design, I highly recommend checking out the technical deep-dives at ** WellAlly Blog**. They cover advanced topics like "Multi-Agent Orchestration" and "Long-context DOM Processing" which were the inspiration for this architecture. 🥑

By combining Playwright's reliability with LLM Function Calling, we've built a system that handles the unpredictability of modern web UIs. This "Agentic" approach is the future of Intelligent Task Automation, turning complex, multi-step workflows into simple natural language prompts.

What's next?

Did you find this helpful? Drop a comment below if you've tried building agents for web automation! 🚀💻