When “Send” Isn’t Risky: Context-Aware Confirmation in an Android AI Agent

A developer extended an Android AI agent's confirmation guard to weigh the foreground app's package name alongside the action label, so that a "send" in Gmail or PayPal triggers a confirmation prompt while the same word in a notes app does not. The design classifies packages into high, medium and low risk tiers and applies different keyword sets to each, using whole-word matching to avoid false positives like "payload" or "sender". The developer argues that over-triggering confirmations trains users to tap through prompts, undermining the guard's real protection.

In the last post I covered the two-layer confirmation guard: the LLM's requires confirmation flag plus a keyword guard as a dumb-but-reliable backstop. A reader raised the right follow-up: "send" in a banking app and "send" in a notes app are not the same risk. The guard shouldn't treat them the same way. This post covers the next layer — making the confirmation guard context-aware by weighing app identity and action type together. A flat list of risky keywords has two failure modes. False negatives the dangerous kind : the guard misses a risky action because the label is in another language, uses a synonym, or comes from a resource ID the guard never sees. False positives the usability kind : the guard fires on "payload", "sender", "shareholder" — or on "send" in an app where sending is so routine it carries no real risk. If confirmations fire too often on benign actions, users train themselves to tap through without reading. When the real confirmation arrives, it gets tapped through too. The second failure mode undermines the first. A guard that cries wolf enough times stops being a guard. The AgentStepRequest already carries current package — the package name of the app currently in focus: class AgentStepRequest BaseModel : task: str screen description: str current package: Optional str = None // e.g. "com.google.android.gm" ... This is the signal that makes context-aware risk assessment possible. The same action means different things in different apps. enum class AppRiskTier { HIGH, // Banking, payments, email clients, messaging MEDIUM, // Social, productivity, calendar LOW // Notes, utilities, entertainment } object AppRiskClassifier { private val HIGH RISK PACKAGES = setOf "com.google.android.gm", // Gmail "com.android.mms", // SMS "org.telegram.messenger", // Telegram "com.whatsapp", // WhatsApp "com.paypal.android.p2pmobile", // PayPal "com.google.android.apps.nbu.paisa.user" // GPay private val LOW RISK PACKAGES = setOf "com.google.android.keep", // Keep Notes "com.spotify.music", // Spotify "com.google.android.youtube" // YouTube fun tierFor packageName: String? : AppRiskTier { if packageName == null return AppRiskTier.MEDIUM return when { HIGH RISK PACKAGES.any { packageName.startsWith it } - AppRiskTier.HIGH LOW RISK PACKAGES.any { packageName.startsWith it } - AppRiskTier.LOW else - AppRiskTier.MEDIUM } } } object ConfirmationGuard { private val HIGH RISK HINTS = setOf "send", "pay", "transfer", "purchase", "buy", "confirm payment", "place order", "checkout" private val MEDIUM RISK HINTS = setOf "delete", "remove", "uninstall", "submit", "sign out", "log out", "deactivate", "cancel subscription", "share" private val LOW RISK HINTS = setOf "post", "publish", "save" fun shouldConfirm text: String?, packageName: String? : Boolean { if text == null return false val lower = text.lowercase .trim val tier = AppRiskClassifier.tierFor packageName return when tier { AppRiskTier.HIGH - HIGH RISK HINTS.any { matchesWord lower, it } || MEDIUM RISK HINTS.any { matchesWord lower, it } AppRiskTier.MEDIUM - HIGH RISK HINTS.any { matchesWord lower, it } || MEDIUM RISK HINTS.any { matchesWord lower, it } AppRiskTier.LOW - HIGH RISK HINTS.any { matchesWord lower, it } } } // Whole-word match to avoid "pay" in "payload", "send" in "sender" private fun matchesWord text: String, keyword: String : Boolean { val pattern = Regex "\\b${Regex.escape keyword }\\b" return pattern.containsMatchIn text } } Two changes from the flat version: whole-word matching instead of substring matching, and threshold variation by app tier. "Send" in a notes app LOW tier doesn't trigger confirmation. "Send" in Gmail HIGH tier does. A reader flagged the language gap in the last post — a phone set to French or Arabic renders "Envoyer" or "إرسال", neither of which matches "send". The visible label isn't reliable across locales. The fix: use the view's contentDescription or resource ID alongside the label. Resource IDs are set by the developer and almost always stay in English regardless of locale: fun shouldConfirm visibleText: String?, contentDescription: String?, resourceId: String?, packageName: String? : Boolean { // Check visible label, contentDescription, and resource ID // Resource IDs travel across locales; visible text does not return listOf visibleText, contentDescription, resourceId .any { text - shouldConfirmText text, packageName } } The Accessibility API surfaces all three. Using all three means the guard fires correctly regardless of the device language. enum class ConfirmationSource { LLM FLAGGED, KEYWORD GUARD, CONTEXT AWARE GUARD, BOTH } CONTEXT AWARE GUARD in the log means the flat guard would have missed it LOW tier, medium-risk keyword or the flat guard would have over-fired LOW tier, low-risk keyword suppressed . Both are worth tracking separately — one shows where context is adding value, the other shows where it's reducing noise. A confirmation guard that fires on everything trains users to ignore it. A guard that knows which app it's running in fires less often and means more when it does. Context doesn't weaken the protection — it concentrates it where the risk is real. Series so far: