181–190 of 213 posts
It’s not obvious but you can use this with your own local (or any) models. https://support.mozilla.org/en-US/kb/smart-window-byom
Earlier quoted context omitted.
Mozilla has been in a bit of a recursive feedback loop death spiral for years now of: Browser loses market share -> try some weird thing -> very few people like or use it -> browser loses market share -> management says "oh shit we're losing market share we better try some weird thing" I say this as a person that uses firefox with ublock origin 99.5% of the time. It's better in my opinion than Chrome. I can at least…
This is the confidently incorrect narrative that gets repeated in the comment sections practically every time Mozilla is mentioned but it's every bit as incorrect now as it's been all the previous times. So here we go again for the millionth time: the big losses of Mozilla market share were approximately during the 2010s. The era of side bets on unique features is approximately the 2020s. The unique features didn't r…
Earlier quoted context omitted.
I am not trying to defend Mozilla doing this, and I don't support sending data to cloud based services like this in a way that users won't understand. But I also think that the state of the art in small LLM and user device capabilities aren't there yet to put a "good enough to be actually useful" local-only LLM as a prepackaged thing in a mass market distributed browser. You don't want a browser that takes 10GB of ex…
> that it's not some big American cloud company but a relatively small startup in France How it's any better? Small companies can be bought by big companies. French government is as capable of trampling over their citizen's privacy the moment they feel they need it as US government is. And putting a squeeze on a small startup is way easier than on a major cloud company (not that either is particularly hard). Also, Op…
This. I've come to view a startup as a company without a business model, doing everything to get acquired by a mega corp that will finally squeeze the juice out of the userbase
(The lack of a viable business model applies to some mega corps too)
People here can be privacy aware and well informed and avoid these data slurps-- perhaps go dig up the hidden settings to completely disable it or add firewalls so an errant keypress won't upload your browsing history. That's good.
But anything I share with another person or put on some webpage for another-- perhaps highly trusted person like a doctor or lawyer-- is exposed to them uploading it perhaps completely unwittingly (e.g. they fell for the exaggerated privacy claims) or due to an innocent misclick.
Normalizing privacy fails like this undermines the ability of even the most informed and aware people to opt-out.
Earlier quoted context omitted.
> This is an excellent use case for completely local, small model inference Is it really ? LLM take lot of ram and drain battery. People run Firefox on low end computer.
Indeed. Realistically a 'capable' small local LLM, even one that's definitely not as good as externally hosted ones will require a single 16GB GPU and access to basically all of the RAM on the GPU. That's not something people running Firefox on a $500 laptop with 8 or 16GB of total system RAM and a CPU-integrated basic graphics system have to spare.
On a 8GB system with a SSD it might be a bit slow due to having to keep experts on disk, but perfectly usable. On fast systems it may well be faster than the network round trip for many tasks.
Earlier quoted context omitted.
> Nowhere in the news they say it is offline Reading HN, I 100% understood “private” to be marketing copy for “local inference”. How else could the information be kept private? I begrudgingly suppose that there is precedent for the usage, like Apple’s Private Relay, but I really don’t think they are comparable.
> Reading HN, I 100% understood “private” to be marketing copy for “local inference”. Pretty ironic then that here, Mozilla is specifically labeling their "we send it all to the cloud" service as "private."
This is an excellent use case for completely local, small model inference, yet for inexplicable reasons Mozilla wants to normalize uploading your entire private browsing history to a cloud. These (Mistral's and Mozilla's) marketing pages aren't candid enough to clearly explain the difference between local and cloud inference, and that they're asking you to consent to enabling the latter. I'd call that the bare minimu…
I am not trying to defend Mozilla doing this, and I don't support sending data to cloud based services like this in a way that users won't understand. But I also think that the state of the art in small LLM and user device capabilities aren't there yet to put a "good enough to be actually useful" local-only LLM as a prepackaged thing in a mass market distributed browser. You don't want a browser that takes 10GB of ex…
It’s not obvious but you can use this with your own local (or any) models. https://support.mozilla.org/en-US/kb/smart-window-byom
I tried adding my own endpoint, but the save button was still greyed out so it did not work. Maybe the feature is not fully implemented yet? In about:config, I could add the endpoint and api key, but not the model name.
Earlier quoted context omitted.
Indeed. Realistically a 'capable' small local LLM, even one that's definitely not as good as externally hosted ones will require a single 16GB GPU and access to basically all of the RAM on the GPU. That's not something people running Firefox on a $500 laptop with 8 or 16GB of total system RAM and a CPU-integrated basic graphics system have to spare.
No it doesn't. TODAY you can run with good performance a local LLM entirely on CPU that can do translation, summaries, search assistance, and all the things mentioned here. On a 8GB system with a SSD it might be a bit slow due to having to keep experts on disk, but perfectly usable. On fast systems it may well be faster than the network round trip for many tasks.
What specific LLM and how much RAM does it take up to load? How does it run on somebody's 8GB RAM $600 laptop they bought from Best Buy? What pp/s and token/s rate on that hardware?
Earlier quoted context omitted.
You can control this in the settings: - Learn from chats in Smart Window - Learn from browsing in Classic and Smart Windows
[flagged]