When “Send” Isn’t Risky: Context-Aware Confirmation in an Android AI Agent

In the last post I covered the two-layer confirmation guard: the LLM's requires_confirmation flag plus a keyword guard as a dumb-but-reliable backstop. A reader raised the right follow-up: "send" in a banking app and "send" in a notes app are not the same risk. The guard shouldn't treat them the same way.

This post covers the next layer — making the confirmation guard context-aware by weighing app identity and action type together.

A flat list of risky keywords has two failure modes.

False negatives (the dangerous kind): the guard misses a risky action because the label is in another language, uses a synonym, or comes from a resource ID the guard never sees.

False positives (the usability kind): the guard fires on "payload", "sender", "shareholder" — or on "send" in an app where sending is so routine it carries no real risk. If confirmations fire too often on benign actions, users train themselves to tap through without reading. When the real confirmation arrives, it gets tapped through too.

The second failure mode undermines the first. A guard that cries wolf enough times stops being a guard.

The AgentStepRequest already carries current_package — the package name of the app currently in focus:

class AgentStepRequest(BaseModel):
    task: str
    screen_description: str
    current_package: Optional[str] = None  // e.g. "com.google.android.gm"
    ...

This is the signal that makes context-aware risk assessment possible. The same action means different things in different apps.

enum class AppRiskTier {
    HIGH,    // Banking, payments, email clients, messaging
    MEDIUM,  // Social, productivity, calendar
    LOW      // Notes, utilities, entertainment
}

object AppRiskClassifier {

    private val HIGH_RISK_PACKAGES = setOf(
        "com.google.android.gm",           // Gmail
        "com.android.mms",                  // SMS
        "org.telegram.messenger",           // Telegram
        "com.whatsapp",                     // WhatsApp
        "com.paypal.android.p2pmobile",    // PayPal
        "com.google.android.apps.nbu.paisa.user" // GPay
    )

    private val LOW_RISK_PACKAGES = setOf(
        "com.google.android.keep",         // Keep Notes
        "com.spotify.music",               // Spotify
        "com.google.android.youtube"       // YouTube
    )

    fun tierFor(packageName: String?): AppRiskTier {
        if (packageName == null) return AppRiskTier.MEDIUM
        return when {
            HIGH_RISK_PACKAGES.any { packageName.startsWith(it) } -> AppRiskTier.HIGH
            LOW_RISK_PACKAGES.any { packageName.startsWith(it) } -> AppRiskTier.LOW
            else -> AppRiskTier.MEDIUM
        }
    }
}
object ConfirmationGuard {

    private val HIGH_RISK_HINTS = setOf(
        "send", "pay", "transfer", "purchase", "buy",
        "confirm payment", "place order", "checkout"
    )

    private val MEDIUM_RISK_HINTS = setOf(
        "delete", "remove", "uninstall", "submit",
        "sign out", "log out", "deactivate",
        "cancel subscription", "share"
    )

    private val LOW_RISK_HINTS = setOf(
        "post", "publish", "save"
    )

    fun shouldConfirm(
        text: String?,
        packageName: String?
    ): Boolean {
        if (text == null) return false
        val lower = text.lowercase().trim()
        val tier = AppRiskClassifier.tierFor(packageName)

        return when (tier) {
            AppRiskTier.HIGH ->
                HIGH_RISK_HINTS.any { matchesWord(lower, it) } ||
                MEDIUM_RISK_HINTS.any { matchesWord(lower, it) }

            AppRiskTier.MEDIUM ->
                HIGH_RISK_HINTS.any { matchesWord(lower, it) } ||
                MEDIUM_RISK_HINTS.any { matchesWord(lower, it) }

            AppRiskTier.LOW ->
                HIGH_RISK_HINTS.any { matchesWord(lower, it) }
        }
    }

    // Whole-word match to avoid "pay" in "payload", "send" in "sender"
    private fun matchesWord(text: String, keyword: String): Boolean {
        val pattern = Regex("\\b${Regex.escape(keyword)}\\b")
        return pattern.containsMatchIn(text)
    }
}

Two changes from the flat version: whole-word matching instead of substring matching, and threshold variation by app tier. "Send" in a notes app (LOW tier) doesn't trigger confirmation. "Send" in Gmail (HIGH tier) does.

A reader flagged the language gap in the last post — a phone set to French or Arabic renders "Envoyer" or "إرسال", neither of which matches "send". The visible label isn't reliable across locales.

The fix: use the view's contentDescription or resource ID alongside the label. Resource IDs are set by the developer and almost always stay in English regardless of locale:

fun shouldConfirm(
    visibleText: String?,
    contentDescription: String?,
    resourceId: String?,
    packageName: String?
): Boolean {
    // Check visible label, contentDescription, and resource ID
    // Resource IDs travel across locales; visible text does not
    return listOf(visibleText, contentDescription, resourceId)
        .any { text ->
            shouldConfirmText(text, packageName)
        }
}

The Accessibility API surfaces all three. Using all three means the guard fires correctly regardless of the device language.

enum class ConfirmationSource {
    LLM_FLAGGED,
    KEYWORD_GUARD,
    CONTEXT_AWARE_GUARD,
    BOTH
}

CONTEXT_AWARE_GUARD in the log means the flat guard would have missed it (LOW tier, medium-risk keyword) or the flat guard would have over-fired (LOW tier, low-risk keyword suppressed). Both are worth tracking separately — one shows where context is adding value, the other shows where it's reducing noise.

A confirmation guard that fires on everything trains users to ignore it. A guard that knows which app it's running in fires less often and means more when it does. Context doesn't weaken the protection — it concentrates it where the risk is real.

Series so far: