Minim: Privacy-Aware Minimal View for Agents via Trusted Local Sanitization

Researchers propose MINIM, a trusted local broker that performs privacy-aware minimization on client-side UI observations before transmission to remote LLM inference servers. The system learns dual scores for each UI element—sensitivity and task necessity—to selectively keep, abstract, or remove content, reducing sensitive leakage while preserving task-critical information. Experiments on WebArena show MINIM effectively prunes irrelevant sensitive data without compromising agent action reliability.

arXiv:2606.13949v1 Announce Type: new Abstract: Modern LLM-powered autonomous agents increasingly rely on rich user interface UI state observations to achieve reliable action grounding in complex digital environments. However, many deployments transmit the full UI state to remote inference servers even when most elements are irrelevant to the current task, which can leak sensitive but unnecessary context such as authentication codes, private notifications, and background application states. We propose MINIM, a trusted local broker that performs privacy-aware minimization on the client side before any observation leaves the device. Grounded in Contextual Integrity CI , MINIM learns a dual-score representation for each UI element by predicting an inherent sensitivity score s and a task-conditioned necessity score n . These scores drive a ternary disclosure policy that keeps essential elements, abstracts sensitive attributes when needed, and removes task-irrelevant content. We optimize a CI-aware objective that penalizes necessity errors more strongly on high-risk content, enabling aggressive pruning while preserving task-critical information. Experiments on real-world UI observations derived from WebArena show that MINIM substantially reduces task-irrelevant sensitive leakage while preserving task-critical semantic context and the interactive affordances required for reliable agent actions.