{"slug": "project-log-15-my-ai-agent-just-completed-its-first-multi-app-task", "title": "Project Log #15: My AI Agent Just Completed Its First Multi-App Task", "summary": "A developer building an AI phone agent achieved a milestone: the agent completed its first multi-app task by copying an account balance from a banking app and sending it via WhatsApp. The key enabler was a task memory system using a persistent key-value store that bridges data between apps. Challenges remain with app loading times, OCR accuracy, and screen state management.", "body_md": "Day 15. The agent switched between two apps, copied data from one, and pasted it into another.\n\nFor 14 days, my AI agent could do one thing well: operate within a single app. Open WhatsApp. Find a contact. Send a message. All within one ecosystem.\n\nToday, that changed.\n\n**The Milestone**\n\nI gave the agent a command: \"Copy my account balance from my banking app and send it to Mom on WhatsApp.\"\n\nIt opened the banking app. It read the balance from the screen using OCR (since banking apps have zero accessibility labels—F score, remember?). It copied the number. It switched to WhatsApp. It found Mom. It typed the balance. It hit send.\n\nTwo apps. One task. Completed autonomously.\n\n**What Made This Possible**\n\nThe key was building a task memory system. The agent now stores values extracted from one app in a simple key-value store (a Python dictionary that persists for the duration of the task). When it switches to the next app, it retrieves that value and uses it.\n\nThe flow works like this:\n\n`account_balance`\n\n.`account_balance`\n\nfrom memory. Type it. Send.The memory system is simple—just a dictionary that gets passed between steps. But it's the bridge between single-app automation and true multi-app workflows.\n\n**Today's Progress**\n\n| Task | Status |\n|---|---|\n| Built task memory system (key-value store) | ✅ Done |\n| Tested \"Copy balance → send to Mom\" | ✅ Success |\n| Tested \"Copy address → paste into Maps\" | ✅ Success |\nUpdated `agent.py` with multi-app support |\n✅ Done |\n| Added task memory to README | ✅ Done |\n\n**What Still Breaks**\n\nThe banking app is slow to load on my phone. Sometimes the agent times out waiting for the balance screen to appear. I added a retry loop, but it's not perfect.\n\nApp switching is clunky. The agent uses `adb shell am start`\n\nto launch apps, but if the app was already open in the background, it sometimes opens to the wrong screen. I'm working on a \"reset to home screen\" step between app switches.\n\nAnd the OCR on banking app text is unreliable. The font is small. The contrast is low. The agent sometimes misreads the balance by a digit or two. For a task like \"send my balance to Mom,\" that's embarrassing but not catastrophic. For a task like \"transfer money,\" that's dangerous. I need higher accuracy before I can trust this with anything financial.\n\n**The Bigger Picture**\n\nMulti-app workflows are the difference between a cool prototype and a useful tool. An agent that can only operate in one app is a novelty. An agent that can move data between apps is an assistant.\n\nThe next step is chaining more complex workflows: \"Find my last three transactions, summarise them, and email them to my accountant.\" That's three apps. That's the goal.\n\n**What's Next (Day 16)**\n\n**The Repo**\n\n👉 **github.com/Dexter2344/phone-agent**\n\n`agent.py`\n\nnow includes a task memory system. Multi-app workflows are supported. README updated with Day 15 status.\n\nThis is Day 15. The agent is no longer trapped in one app.", "url": "https://wpnews.pro/news/project-log-15-my-ai-agent-just-completed-its-first-multi-app-task", "canonical_source": "https://dev.to/okeke_chukwudubem_5f3bf49/project-log-15-my-ai-agent-just-completed-its-first-multi-app-task-38n7", "published_at": "2026-07-11 22:21:17+00:00", "updated_at": "2026-07-11 22:44:40.696884+00:00", "lang": "en", "topics": ["ai-agents", "artificial-intelligence", "computer-vision", "developer-tools"], "entities": ["Dexter2344", "WhatsApp", "phone-agent", "OCR", "Python"], "alternates": {"html": "https://wpnews.pro/news/project-log-15-my-ai-agent-just-completed-its-first-multi-app-task", "markdown": "https://wpnews.pro/news/project-log-15-my-ai-agent-just-completed-its-first-multi-app-task.md", "text": "https://wpnews.pro/news/project-log-15-my-ai-agent-just-completed-its-first-multi-app-task.txt", "jsonld": "https://wpnews.pro/news/project-log-15-my-ai-agent-just-completed-its-first-multi-app-task.jsonld"}}