Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MemoryCache: Augmenting local AI with browser data

future.mozilla.org

Post reply on HN
Print view View original

51–60 of 108 posts

Re: MemoryCache: Augmenting local AI with browser data

#51

Post by csbartus »

Jayakumark wrote: post #2

Was just talking about this on reddit like two days ago Instead of data going to models, we need models come to our data which is stored locally and stay locally. While there are many OSS for Loading personal data, they dont do images or videos. In the future everyone may get their own Model but for now tech is there but product/OSS is missing for everyone to get their own QLORA or RAG or Summarizer. Not just message…

> Instead of data going to models, we need models come to our data which is stored locally and stay locally.

That's the most important idea I've read since ChatGPT / last year.

I'll wait for this. Then build my own private AI. And share it / pair it for learning with other private AIs, like a blogroll.

As always, there will be two 'different' AIs: a.) the mainstream, centralized, ad/revenue-driven, capitalist, political, controlling / exploiting etc. b.) personal, trustworthy, polished on peer networks, fun, profitable for one / a small community.

If by chance, commercial models will be better than open source models, due to better access to computing power / data, please let me know. We can go back to SETI and share our idle computing power / existing knowledge

Re: MemoryCache: Augmenting local AI with browser data

#52

Post by jval43 »

danielovichdk wrote: post #39

My usage with browsing is not relevant for this. I don't want to "chat" with my browsin g history. I would simply love my browser would index my bookmarks on my OS so I could search the actual content of those bookmarks. The feedback loop coming gained from chatgtp will I assume always be way better than my local gpt equivalent. But often I bookmark pages where I know the information on there are important enough for…

Didn't Chrome do this at the very beginning, when it was initially released? I faintly remember that being a feature.

Personally I would already be content if my browsers didn't forget their history all the time, both Firefox and Safari history is way too short-lived.

Re: MemoryCache: Augmenting local AI with browser data

#53

Post by ath3nd »

They might be onto something here.

Instead of doing lots of back-n-forth with the giants, enriching them with each prompt, you get a smaller local model that's much more respectful of your privacy.

That's an operating model I am willing to do some OSS contributions to, or even bankroll.

Gotta love the underdogs, even if admittedly, I am not a big Mozilla org fan.

Re: MemoryCache: Augmenting local AI with browser data

#54

Post by jakderrida »

Jayakumark wrote: post #2

Was just talking about this on reddit like two days ago Instead of data going to models, we need models come to our data which is stored locally and stay locally. While there are many OSS for Loading personal data, they dont do images or videos. In the future everyone may get their own Model but for now tech is there but product/OSS is missing for everyone to get their own QLORA or RAG or Summarizer. Not just message…

> Our browsing history tells a lot about what we read but no one seems to make use of it other than google for ads.. Almost everyone has a habit of reading x news site, x social network, x youtube videos etc.. Ok, here are the summary for you from these 3 today.

I was imagining something a little more ambitious. Like a model that uses our search history and behavior to derive how to best compose a search query. Bing Chat's search queries look like what my uncle would type right after I explained to him what a search engine is. Throw in some advanced operators like site: or filetype: or at least parentheses along with AND/OR. Surely, we can fine tune it to emulate the search processes of the most impressive researchers, paralegals, and teenagers on the spectrum that immediately factcheck your grandpop's Ellis Island story, with evidence he both arrived at first and was naturalized in Chicago.

Re: MemoryCache: Augmenting local AI with browser data

#55

Post by pixxel »

no_time wrote: post #49

Good idea. Mozilla gets a lot of rightful hate for their mishandling of FF and their political preaching, but I believe they are still capable of developing tech that is both privacy preserving and user friendly at the same time. I use the offline translator built into FF regularly and It's magic. I would've never thought something like that can run locally, without a server park worth of hardware thrown at it. Here'…

Well said; I agree wholeheartedly.

Re: MemoryCache: Augmenting local AI with browser data

#56

Post by altairprime »

ath3nd wrote: post #53

They might be onto something here. Instead of doing lots of back-n-forth with the giants, enriching them with each prompt, you get a smaller local model that's much more respectful of your privacy. That's an operating model I am willing to do some OSS contributions to, or even bankroll. Gotta love the underdogs, even if admittedly, I am not a big Mozilla org fan.

It’s what Apple’s been doing for a few years, though it remains unclear how much of that is “AI”. So it makes sense that someone else would enter that niche.

Re: MemoryCache: Augmenting local AI with browser data

#57

Post by SpaceManNabs »

This seems completely overkill.

I don't even like having to clear my history and wtv regularly. I use incognito mode most times.

Now I have monitor what my local AI collects?

"through the lens of privacy" my ass, man.

Why would I ask my browser what the meaning of a life well lived is?

Re: MemoryCache: Augmenting local AI with browser data

#58

Post by visarga »

ath3nd wrote: post #53

They might be onto something here. Instead of doing lots of back-n-forth with the giants, enriching them with each prompt, you get a smaller local model that's much more respectful of your privacy. That's an operating model I am willing to do some OSS contributions to, or even bankroll. Gotta love the underdogs, even if admittedly, I am not a big Mozilla org fan.

In the future their AIs are going to talk to our AIs. Because we need protection.

Re: MemoryCache: Augmenting local AI with browser data

#59

Post by cinntaile »

orbital-decay wrote: post #26
simonw wrote: post #25

Earlier quoted context omitted.

"While there are many OSS for Loading personal data, they dont do images or videos" Local models for images are getting pretty good. LLaVA is an LLM with multi-modal image capabilities that runs pretty well on my laptop: https://simonwillison.net/2023/Nov/29/llamafile/ Models like Salesforce BLIP can be used to generate captions for images too - I built a little CLI tool far that here: https://github.com/simonw/blip-…

CogVLM blows LLaVA out of the water, although it needs a beefier machine (quantized low-res version barely fits into 12GB VRAM, not sure about the accuracy of that).

I have no actual knowledge in this area so I'm not sure if it's entirely relevant but an update from the 7th of December on the CogVLM repo says it now works with 11GB of VRAM.

Re: MemoryCache: Augmenting local AI with browser data

#60

Post by amelius »

Jayakumark wrote: post #2

Was just talking about this on reddit like two days ago Instead of data going to models, we need models come to our data which is stored locally and stay locally. While there are many OSS for Loading personal data, they dont do images or videos. In the future everyone may get their own Model but for now tech is there but product/OSS is missing for everyone to get their own QLORA or RAG or Summarizer. Not just message…

Local compute is so 80s, when people moved away from dumb terminals and mainframes, to PCs.
Post reply on HN