Live data from Hacker News

MemoryCache: Augmenting local AI with browser data

future.mozilla.org

41–50 of 108 posts

Re: MemoryCache: Augmenting local AI with browser data

#41

Classic bookmarks have failed because mnemonic organization doesn't scale. This kind of interface does, and can replace it entirely if done right. Thinking of it, something like this can be used for all your local files as well, acting as a better version of the old filesystem-as-a-database idea. Or for a specific knowledge base (think LLM-powered Zotero).

Sounds like you just invented the modern version of Windows Longhorn

Re: MemoryCache: Augmenting local AI with browser data

#44
post #18

Earlier quoted context omitted.

I assume that training LLMs locally require high-end hardware. Even running a model requires a decent CPU or, even better, a high end GPU, but it is not so expensive as training a model. And usually you have to use hardware that is available on the cloud, so not much of privacy here.

You don't need to train the model on your data: you can use retrieval augmented generation to add the relevant documents to your prompt at query time.

Thank you for explanation. I see there is still a lot I have to learn about LLMs.

Re: MemoryCache: Augmenting local AI with browser data

#45

My usage with browsing is not relevant for this. I don't want to "chat" with my browsin g history. I would simply love my browser would index my bookmarks on my OS so I could search the actual content of those bookmarks. The feedback loop coming gained from chatgtp will I assume always be way better than my local gpt equivalent. But often I bookmark pages where I know the information on there are important enough for…

Small data sets suffer from bad recall in full text search. So a bit of smart fuzzyness added to the search by AI could improve the experience on locally indexed bookmarks quite well.

Re: MemoryCache: Augmenting local AI with browser data

#46
post #9

Very misleading name. The word "Memory" has a distinct meaning in relation to computing, but this is more about human memories.

I was going to ignore this as a troll comment because computer memory has its antecedents in human memory but the commenter is right - the combination of memory and cache to talk about human memory seems misleading.

Re: MemoryCache: Augmenting local AI with browser data

#47
post #31

Earlier quoted context omitted.

Teach it to press the skip ad button

Or it could click "hide" on cookie banners for me!

They actually already added this, but it's still in a limited trial phase.

https://support.mozilla.org/en-US/kb/cookie-banner-reduction

Re: MemoryCache: Augmenting local AI with browser data

#48
post #18

Earlier quoted context omitted.

I assume that training LLMs locally require high-end hardware. Even running a model requires a decent CPU or, even better, a high end GPU, but it is not so expensive as training a model. And usually you have to use hardware that is available on the cloud, so not much of privacy here.

You don't need to train the model on your data: you can use retrieval augmented generation to add the relevant documents to your prompt at query time.

This works if the document plus prompt fit in the context window. I suspect the most popular task for this workflow is summary which presumably means large documents. That's when you begin scaling out to a vector store and implementing those more advanced workflows. It does work even by sending a large document on certain local models, but even with the highest tier MacBook Pro a large document can quickly choke up any LLM and bring inference speed to a crawl. Meaning, a powerful client is still required no matter what. Even if you generate embeddings in "real-time" and dump to a vector store that process would be slow in most consumers hardware.

If you're passing in smaller documents then it works pretty good for real-time feedback.

Re: MemoryCache: Augmenting local AI with browser data

#49
Good idea. Mozilla gets a lot of rightful hate for their mishandling of FF and their political preaching, but I believe they are still capable of developing tech that is both privacy preserving and user friendly at the same time.

I use the offline translator built into FF regularly and It's magic. I would've never thought something like that can run locally, without a server park worth of hardware thrown at it.

Here's hoping this experiment turns out the same way.

Post reply on HN