Google adds Select-from-screen to Gemini in Chrome

Google added a 'Select from screen' feature to Gemini in Chrome, allowing users to draw boxes around text or images in the active tab and attach them to prompts, arriving with Chrome 149. Separately, Gemini 3.5 Flash now includes native 'computer use' capabilities in the Gemini API for building agents that interact with browsers and desktops. The updates aim to make browser-based assistants more context-aware and agentic, with enterprise guardrails for sensitive actions.

What happened Google rolled out a new "Select from screen" capability for Gemini in Chrome , which lets users draw boxes around specific text or images in the active tab and attach those selections to a Gemini prompt, according to Google's support documentation Google Support and reporting from Digital Trends and 9to5Google. Multiple outlets report the feature is arriving with Chrome 149 , and Google's support article documents the UI flow: open Ask Gemini in Chrome, choose Add menu and "Select from screen", then draw selection boxes and submit a prompt Google Support; Digital Trends; 9to5Google . Google's product blog and Chrome AI pages simultaneously describe broader updates to Gemini in Chrome, including Auto browse and new image features such as Nano Banana, and list platform availability in the U.S. for Windows, MacOS, and Chromebook Plus Google blog; Chrome AI page . Technical details Editorial analysis - technical context: The reported "Select from screen" flow converts a user-drawn region into an attached input for the assistant, removing manual screenshot-and-paste steps and creating a deterministic grounding signal for multimodal prompts. Reported coverage shows the feature is surfaced from the side-panel Ask Gemini UI and the bottom of the plus menu, which implies a tight integration between Chrome's frontend and the assistant prompt state Digital Trends; Google Support . Separately, 9to5Google and Google's blog report that Gemini 3.5 Flash now exposes native "computer use" capabilities in the Gemini API, replacing a previously separate model for agentic browser and app interactions; Google's materials say developers can "build custom agents that can see, reason and take action across browser, mobile and desktop environments" 9to5Google; Google blog . The published safety notes reported by 9to5Google indicate enterprise controls such as explicit user confirmation for sensitive actions and automatic stops on detected indirect prompt injection attempts. Context and significance Public reporting frames these changes as part of a larger trend to make browser-based assistants more context-aware and agentic. By lowering friction for supplying exact visual context to prompts, the feature aligns with broader moves toward embedding multimodal grounding directly into user workflows, rather than relying on manual copy-paste or separate screenshot tools Digital Trends; 9to5Google . For developers, a native computer use capability inside Gemini 3.5 Flash consolidates perception-action primitives into the model-side API rather than forcing separate orchestration layers; reporters note Google positions this for "continuous software testing and knowledge work across professional applications" 9to5Google . At the same time, Google's documentation and 9to5Google reporting highlight built-in enterprise guardrails, which address some operational risk vectors but do not eliminate the need for org-level governance. What to watch For practitioners: track documentation and API examples from Google for Gemini 3.5 Flash availability and quota , client-side permission flows in Chrome how selections are consented and logged , and the rollout schedule beyond the initial U.S. Windows/MacOS/Chromebook Plus availability reported in Google's blog and Chrome pages. Observers should watch whether selection metadata page URL, DOM context is attached to prompts and how Google documents retention, telemetry, and enterprise controls. Also monitor whether similar selection-first interfaces appear in other browsers or as cross-platform SDK features in the Gemini API, and whether third-party developers adopt the new computer use primitives for multi-step automation use cases Google Support; Google blog; 9to5Google . Scoring Rationale The integration of direct screen selection into Gemini in Chrome and the computer use capability in Gemini 3.5 Flash materially lower friction for multimodal prompts and agentic tasks, creating practical opportunities for automation and testing workflows. This is a notable product-level update for developers and platform teams, though not a frontier-model breakthrough. Practice with real Ad Tech data 90 SQL & Python problems · 15 industry datasets Active Search Campaigns by BudgetEasy /problems/sql/active-search-campaigns-by-budget High CPC Clicks & Poor Landing PagesMedium /problems/sql/high-cpc-clicks-poor-landing-page Campaign ROAS by Attribution ModelHard /problems/sql/campaign-roas-by-attribution-model 250 free problems · No credit card See all Ad Tech problems /problems/datasets/adtech