It may be polarizing, but there are lot of people using AI right now. Most of those users are likely exclusive to a chatbot, like ChatGPT or Gemini, using it as a Google replacement (don't do that), or as a confidant (please don't do that either). But if your goal is to use AI as a personal assistant, listen up: On Wednesday, Google announced two new updates for Gemini on macOS, one of which may have some privacy and security implications for users—assuming they opt in to it.
Gemini on macOS gets enhanced dictation #
If you use dictation on macOS (i.e., speaking words out loud and having the AI type them for you), you might have a reason to use Gemini over Apple's built-in solutions. Going forward, Gemini on macOS has "intelligent dictation," which Google says polishes text "instantly at the cursor." That means Gemini will remove your stammers, like "ums" and "ahs" in between words, things you never would type out yourself. This works in any window on your Mac: You just press the Fn key, and start speaking. We'll have to see this feature in action to know how well it works, but it does build off of similar enhancements Gemini has on other platforms. It's a small adjustment, but one that will make it possible to use Gemini to draft texts, emails, or documents without having to totally self-edit. That's the kind of utility I'm looking for from AI features. And since it's enabled by default, everyone using Gemini on Mac will be using it—unless they dig into their settings to turn it off.
Gemini can now "understand" your Mac's display #
Google's second update for Gemini on macOS isn't so straightforward, nor is it enabled by default. Google calls it "screen-aware reasoning," a feature that grants Gemini permission to "understand" what's happening on your Mac's screen. The idea is to hand tasks over to Gemini: If you have your Notes app open, for example, you could press the Fn key and say "Turn these notes into an executive summary for the meeting this afternoon." Gemini will reportedly know to look at your Notes app, analyze your text, and configure it on your behalf, all without needing to work through a proprietary app.
You can stack together a series of commands, too: You could ask Gemini to draft an email for a team dinner, and pull from a select group of files on your Mac for important info (participants, budget requirements, dietary restrictions, etc.) You could continue on, asking Gemini to search Google Maps for specific types of restaurants, and include them as options for the team. And at any point, you can correct yourself, and Gemini will adjust accordingly. For example, if you said the team dinner should be at 6:30 p.m. on Friday, but later on in your command you say, "Sorry, dinner should be 7 p.m." Gemini should make the appropriate changes.
Sure, there's a usefulness to this type of feature. If you're already using Gemini on macOS, letting "see" your Mac's display makes it more helpful. Rather than rely on the Gemini app for all your tasks, you can expand its range to your whole Mac, and, if Google is correct, in quite a complex way. But there's always a privacy risk in allowing AI programs full access to what's happening on your personal computer, and perhaps a security risk as well. Do you want Google's AI to be able to see everything on your screen?
It's not as big a risk as a true agentic program, of course. If you pay for Google's most expensive AI plan, you can access Gemini Spark, which can run autonomously on your Mac. The biggest flag to note is "prompt injection," in which hackers hide malicious commands for your AI in websites. If you task the AI to do something, and it stumbles upon such a command, it will think the request comes from you, not from the hacker. So, when it processes a request like "open [malicious site] and download [malicious file]" it will without question; or "open [crypto wallet], log in with saved credentials, and send funds to [account]," it will follow suit. That shouldn't be an issue with Gemini as it stands, though it theoretically could, if you ask it to perform a task that exposes it to prompt injection.
If you're someone who is interested in the latest features, this isn't to dissuade you from using Gemini's new agentic feature on macOS, but it's worth considering the very real risks before you do.