Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral X Mozilla: Private, Multilingual AI Browsing

mistral.ai

Post reply on HN
Print view View original

151–160 of 211 posts

Re: Mistral X Mozilla: Private, Multilingual AI Browsing

#151

Post by [deleted] »

JeremyNT wrote: post #56
fn-mote wrote: post #22

Earlier quoted context omitted.

> Nowhere in the news they say it is offline Reading HN, I 100% understood “private” to be marketing copy for “local inference”. How else could the information be kept private? I begrudgingly suppose that there is precedent for the usage, like Apple’s Private Relay, but I really don’t think they are comparable.

> Reading HN, I 100% understood “private” to be marketing copy for “local inference”. Pretty ironic then that here, Mozilla is specifically labeling their "we send it all to the cloud" service as "private."

[deleted]

Re: Mistral X Mozilla: Private, Multilingual AI Browsing

#152

Post by redox99 »

KronisLV wrote: post #118
peri-cl wrote: post #5

This is an excellent use case for completely local, small model inference, yet for inexplicable reasons Mozilla wants to normalize uploading your entire private browsing history to a cloud. These (Mistral's and Mozilla's) marketing pages aren't candid enough to clearly explain the difference between local and cloud inference, and that they're asking you to consent to enabling the latter. I'd call that the bare minimu…

> This is an excellent use case for completely local, small model inference I remember using a small model for autocomplete, I think it was Qwen 2.5 Coder 3B way back, on a laptop with 32 GB of RAM and a 13th Gen Intel i7 CPU. It made the fans spin up and loaded things so much (Ollama back then) that it was largely unusable for doing software development at the same time. I don't want inference anywhere near my perso…

Not everyone uses laptops. I can run Qwen 3.8 27B (which is a REALLY capable model) in the background coding for me while I'm simultaneously browsing the web and playing VALORANT without any performance impact, and that's on a 6 year old GPU.

Re: Mistral X Mozilla: Private, Multilingual AI Browsing

#154

Post by walrus01 »

glenstein wrote: post #134
walrus01 wrote: post #59

Earlier quoted context omitted.

Mozilla has been in a bit of a recursive feedback loop death spiral for years now of: Browser loses market share -> try some weird thing -> very few people like or use it -> browser loses market share -> management says "oh shit we're losing market share we better try some weird thing" I say this as a person that uses firefox with ublock origin 99.5% of the time. It's better in my opinion than Chrome. I can at least…

This is the confidently incorrect narrative that gets repeated in the comment sections practically every time Mozilla is mentioned but it's every bit as incorrect now as it's been all the previous times. So here we go again for the millionth time: the big losses of Mozilla market share were approximately during the 2010s. The era of side bets on unique features is approximately the 2020s. The unique features didn't r…

I never said that Chrome and Edge don't have a huge distribution advantage due to being installed by default on peoples' devices. That's also certainly a huge factor in the mass adoption of Chrome as what people consider "the web browser. Or Safari as default browser in MacOS of course.

Re: Mistral X Mozilla: Private, Multilingual AI Browsing

#155

Post by doctorpangloss »

walrus01 wrote: post #47
peri-cl wrote: post #5

This is an excellent use case for completely local, small model inference, yet for inexplicable reasons Mozilla wants to normalize uploading your entire private browsing history to a cloud. These (Mistral's and Mozilla's) marketing pages aren't candid enough to clearly explain the difference between local and cloud inference, and that they're asking you to consent to enabling the latter. I'd call that the bare minimu…

I am not trying to defend Mozilla doing this, and I don't support sending data to cloud based services like this in a way that users won't understand. But I also think that the state of the art in small LLM and user device capabilities aren't there yet to put a "good enough to be actually useful" local-only LLM as a prepackaged thing in a mass market distributed browser. You don't want a browser that takes 10GB of ex…

Every ounce of RAM and spare cycle should be used.

Re: Mistral X Mozilla: Private, Multilingual AI Browsing

#156

Post by tyre »

peri-cl wrote: post #5

This is an excellent use case for completely local, small model inference, yet for inexplicable reasons Mozilla wants to normalize uploading your entire private browsing history to a cloud. These (Mistral's and Mozilla's) marketing pages aren't candid enough to clearly explain the difference between local and cloud inference, and that they're asking you to consent to enabling the latter. I'd call that the bare minimu…

> yet for inexplicable reasons

Come on, man, I’m going to choose you’re just Posting saying shit like this.

Because it’s useful. Because almost nobody has the hardware to run a local model with nearly reasonable performance (in intelligence + speed + RAM usage).

Not everything is perfect. At least they’re building things.

Re: Mistral X Mozilla: Private, Multilingual AI Browsing

#157

Post by tyre »

smsm42 wrote: post #139
well_ackshually wrote: post #88

Earlier quoted context omitted.

The only people having an expectation of translations being done locally is exactly the nerds that keep whining that it's not using a local model. Every single normal person, when presented with a "translate" button either know it's going online, or don't care about it. Begging the purists to run away from Firefox at this point so they can stop wasting everyone's time. Your demands for examplarity and whining about m…

You realize that people that don't care won't even know what Firefox is? Being a niche player and scaring away niche people is the most stupid strategy possible.

Being a small player and throwing away any hope of expanding your user base by building for people who will complain no matter what you do is, well, something!

Re: Mistral X Mozilla: Private, Multilingual AI Browsing

#158

Post by tyre »

ryukoposting wrote: post #115
walrus01 wrote: post #47

Earlier quoted context omitted.

I am not trying to defend Mozilla doing this, and I don't support sending data to cloud based services like this in a way that users won't understand. But I also think that the state of the art in small LLM and user device capabilities aren't there yet to put a "good enough to be actually useful" local-only LLM as a prepackaged thing in a mass market distributed browser. You don't want a browser that takes 10GB of ex…

I'm worried that the middle could fall out of the computing market across the board. If you can afford to keep up with the upgrade treadmill, you'll get private, local inference capabilities. If you can't afford to stay on the treadmill, you'll be stuck with whatever cloudshit malware Silicon Valley wants to foist on you. I acknowledge that this is already the case, to some extent. The cheapest laptops at Best Buy ar…

This has always been the case, forever. You have to pay for a product or service. How you do so can be with cash or your data/body/vote/eyeballs/indirect discretionary purchases.

The amount of work that can be done funded by foundations and free work is nowhere close to what people want.

Re: Mistral X Mozilla: Private, Multilingual AI Browsing

#159

Post by _ache_ »

tyre wrote: post #156
peri-cl wrote: post #5

This is an excellent use case for completely local, small model inference, yet for inexplicable reasons Mozilla wants to normalize uploading your entire private browsing history to a cloud. These (Mistral's and Mozilla's) marketing pages aren't candid enough to clearly explain the difference between local and cloud inference, and that they're asking you to consent to enabling the latter. I'd call that the bare minimu…

> yet for inexplicable reasons Come on, man, I’m going to choose you’re just Posting saying shit like this. Because it’s useful. Because almost nobody has the hardware to run a local model with nearly reasonable performance (in intelligence + speed + RAM usage). Not everything is perfect. At least they’re building things.

The strategies of Google and Apple, regarding how to provide a LLM, seam to disagree with you. Gemini run on a potato and Apple is local first.

So, you may actually have very good performance with local model. Just not yet on *every* device. So the Mozilla strategy here feel very reasonable. A Cloud provider specialised in local models, to be able to switch once local models will be quick enough on most devices.

Re: Mistral X Mozilla: Private, Multilingual AI Browsing

#160

Post by nullc »

walrus01 wrote: post #110
nullc wrote: post #109

Earlier quoted context omitted.

Have you tried Ling-3.0-tiny? It runs fine on CPU-- on a 14700KF gets 40tg/s and 250pp/s and on a ordinary gpu (RTX 4070) does over 200tg/s with no MTP and 7185pp/s. (my figures are Q8, though presumably a good Q4 would be faster) It's certainly not as capable as something that needs a high memory gpu for quick performance, but I was quite impressed with it for what it is. (and fwiw, I had it translate your last para…

How much RAM does it take up in total? I'll have to give that a try on one of my test systems. Looking at a somewhat randomly chose GGUF quantization of it, looks like just under 5GB on disk in Q4, so RAM usage somewhere around 5-6GB? https://huggingface.co/bartowski/Ling-3.0-tiny-GGUF

That sounds about right for Q4.

it's an extremely sparse MOE, so there is some odds of acceptable performance using a smaller in-memory cache and the rest on flash. ... I don't have a setup to test that right now.

(Of course, if translation is all you want much smaller models will work. Ling-tiny can do summarization, dom manipulation, scripting, etc. too).

Post reply on HN