How much RAM browser wants to run LLM local processing?
Usually around 5 GB for a 7B 4-bit quantized model.
Opera becomes the first major browser with built-in access to local AI models
51–60 of 139 posts
Re: Opera becomes the first major browser with built-in access to local AI models
#52[flagged]
Re: Opera becomes the first major browser with built-in access to local AI models
#53The question I have is, why should your browser also be the interface to local AI models? Seems totally unrelated to be honest.
The browser is the new OS.
Re: Opera becomes the first major browser with built-in access to local AI models
#54Earlier quoted context omitted.
It makes it possible to do, in half an hour, tasks which were completely unnecessary to do before.
But why use the web browser as local interface for it? Why not a separate, native application?
Why does the address bar go to a search engine?
When you use an LLM, you provide it context. Where do you usually get that context from? Some service, JIRA, whatever, in your browser.
Re: Opera becomes the first major browser with built-in access to local AI models
#55Re: Opera becomes the first major browser with built-in access to local AI models
#56The question I have is, why should your browser also be the interface to local AI models? Seems totally unrelated to be honest.
Opera has been bundling in a mail client, rss reader, calendar app etc for ages now. I also always thought it was weird but Opera people seem to like it, to the point that inspired browsers like Vivaldi also do that.
Re: Opera becomes the first major browser with built-in access to local AI models
#57FYI for those not aware: Vivaldi(1) is the new "true Opera". The original Opera sold in 2016 to "a Chinese consortium"(2)
1: https://en.wikipedia.org/wiki/Vivaldi_(web_browser) 2: https://www.nytimes.com/2016/02/11/business/dealbook/china-o...
Re: Opera becomes the first major browser with built-in access to local AI models
#58Earlier quoted context omitted.
But why use the web browser as local interface for it? Why not a separate, native application?
You missed the joke
I apologize for the mistake in my previous response. It seems that my information about the poster's intent was outdated or inaccurate.
Upon further research, I have found that I misread the word "unnecessarily".
Re: Opera becomes the first major browser with built-in access to local AI models
#59I actually just submitted something quite relevant - a side project we soft-launched today that uses LLMs to analyse and group browser tabs across various browsers and devices. It turns out that it’s super demanding for today’s models, but with some optimisations and experimentation, we made it work pretty well. Running local LLMs might be a heavy order currently to get the job done, but in the near future I don't se…
This is useful. Why the step of involving LLMs though? I note you cluster the tabs and then GPT-4 is involved to name them. But my tab groups don't usually need names - just the icons tells me what I mostly need to know. Could this work locally better using much smaller sentence-transformer models?
Re: Opera becomes the first major browser with built-in access to local AI models
#60FYI for those not aware: Vivaldi(1) is the new "true Opera". The original Opera sold in 2016 to "a Chinese consortium"(2) 1: https://en.wikipedia.org/wiki/Vivaldi_(web_browser) 2: https://www.nytimes.com/2016/02/11/business/dealbook/china-o...
That's a rather xenophobic take to be stating so loudly. Is there any reason to believe that being associated with Chinese intrinsically makes it lesser?
Edit: they're all chromium anyways.