AI is everywhere, and I was mostly sick of it. I’m sick of the slop. I’m sick of super rich people using it to extort everybody. I’m sick of the investment bubbles, the RAM shortage and I could go on and on and on …
But the technology is actually interesting, and since you can easily host your own AI now, I started to have fun with it. The problem was never the technology; it was always big tech companies and ruthless opportunists, who are using the technology in a bad way. - Who would have guessed?
So I wanted to take a look at the AI functions of Firefox. My first impulse was to watch a YouTube video or two - a little show and tell. But the only videos I found, showed how to disable AI in Firefox. - That’s quite telling, isn’t it.
So I did a fresh install of Firefox (Debian has an older ESR-Version) via Flathub and played with it.
And I like what Mozilla has done with AI, mostly.
- On-Device Model Management: Extensions and native features can download and run compatible ML models (via Transformers.js/ONNX) locally, managed via about:addons > On-Device AI. → That’s amazing. Most of the AI in Firefox runs on your own device. No AI cloud provider needed; no privacy compromise(d).
- PDF Image Alt Text Generation: AI creates descriptive alt text for images added to PDFs to improve accessibility for screen readers. → Using on-device models, that’s great for accessibility. Thumbs up, even if I don’t need it.
- Automatic Website Translation: Real-time translation of web pages directly on the device, without cloud processing. → That’s amazing and a real privacy improvement compared to the cloud based services I had to use in the past.
- Sidebar AI Chatbot Providers: Integrates access to various AI assistants in the sidebar, including ChatGPT, Claude, Gemini, Mistral, and Copilot. → The selection isn’t my thing and I’m pretty sure most Firefox users agree. - Firefox tries so hard to be made for the masses, but the masses mostly ignore Firefox. - With a little bit of “about:config”, you can add your own AI Chat, even a self hosted instance of OpenWebUI, so I and it works well. I’m using it to proofread this text without leaving the page for example.
There should be more, but I haven’t tested it.
What do you think about my experiences? Could you accept AI as a technology and benefit from local and self-hosted application or are you so fed up, that you will block anything AI on principle?
3 Likes
I’m curious to see what useful AI features Firefox can come up with. Some interesting uses of AI in a browser might be buttons to:
tell me if this web page looks like a scam (e.g. romance scam, arrest scam) or attack (e.g. phishing, has link to malware) or is asking for PII or inappropriate info (pics). Maybe particularly valuable for children and the elderly.
find other articles like the one in this page, either agreeing or disagreeing or giving more info about same subject
find where the subject of this article is treated in sources I mostly trust, such as Wikipedia or Arch Wiki or manufacturer’s web site or something
find where the subject of this article is being discussed, on the social networks I belong to
sanity-check this article: do the citations exist and the links work, are the quotes accurate, does it fairly represent the sources it cites or links to ?
in all my open tabs and my browsing history for the last 7 days, where is the page that more-or-less said X about subject Y ?
add a link to this page, and a 1-paragraph summary of it, to my: notes app, bookmark app, web site, new post on social media, or email to my friends
do the recommendations in this article apply to anything in my: computer, network, work, school, finances, life ?
right-click and: find more images “similar” to this one
why won’t this page load ? When you get to a certain critical mass of privacy and security measures, it gets hard to figure out what a site is objecting to. VPN ? DNS-blocker in VPN ? Firefox ? Tracker-blocker in FF settings ? Ad-blocker ? Linux ? Location disabled ? WebRTC disabled ? Canvas disabled ? Fact that I reside in Spain ? Bad cookie ? Site down for everyone ?
Yes, most or all of these can be done some other, less convenient way. Copying URL(s), opening a new tab to an LLM, pasting URL(s), writing a prompt. But having buttons for them right in the browser, and pre-written prompts, reduces friction and increases context. Especially important for normal people doing something such as “is this a a scam ?”.
Yes, today’s LLMs can’t do all of this accurately and reliably enough, and there are issues of privacy, resources, etc. But AI will improve.
If the features don’t work, or I don’t like how they’re done, I’ll turn them off. 1 Like
All your use cases are very troublesome, especially if you want privacy.
An AI model does not process a huge amount of data. Context for every request is limited.
For all answers based on public information, the web search tool feeding the AI model context plays a huge role in the result.
For all answers based on private data, there need to be a RAG-System feeding the AI model context. A RAG-System is basically an Indexing and Search engine crated for local data. All data in a RAG-System must be stored there in a certain way. That means every file you want your AI to have knowledge about must be processed by an embedding-model. The result of that processing, must be written to a vector database. - That’s a lot of processing and a lot of storage space.
Also a model capable enough to process such requests would be big and slow - too big and too slow for most hardware. That means no private local AI except for those self-hosting AI on a beefy (> 10.000 $) server.
2 Likes
Actually, if it will be local, I am not against it. But I HATE server-tied AI.
BTW, this or this will work too if you kill internet permission via FlatSeal 1 Like
Everything is adding AI. Browsers have AI… the up and coming Android 17 will have AI… one wonders if there will be one unifying AI to rule them all… and whether or not there will be 15 competing unifying AIs… only for one more unifying AI to be made to unify them all