Live data from Hacker News

MemoryCache: Augmenting local AI with browser data

future.mozilla.org

21–30 of 108 posts

Re: MemoryCache: Augmenting local AI with browser data

#21
post #16

I hope this encourages Mozilla to focus more on page archiving support on the web. I feel as though they missed a huge opportunity by not making it easy to archive pages with DOM snapshots, or easy to snag videos or images. (Go to Instagram and try to right-click -> download the image; you can't.) Would have been a very good way to differentiate from Chrome, as Google wouldn't want that available for Youtube. And "ou…

[deleted]

Re: MemoryCache: Augmenting local AI with browser data

#22
post #16

I hope this encourages Mozilla to focus more on page archiving support on the web. I feel as though they missed a huge opportunity by not making it easy to archive pages with DOM snapshots, or easy to snag videos or images. (Go to Instagram and try to right-click -> download the image; you can't.) Would have been a very good way to differentiate from Chrome, as Google wouldn't want that available for Youtube. And "ou…

Agree, it seems like it’s insanely hard to back up a modern JS-enabled web page in a usable way that results in a single file which can be easily shared.

Have you tried SingleFile? It sounds like what you’re looking for:

https://github.com/gildas-lormeau/SingleFile

Re: MemoryCache: Augmenting local AI with browser data

#24

I'm confused by the example they gave. > What is the meaning of a life well-lived? Is the response to this based on browser data? Based on the description I was expecting queries more like: > What was the name of that pdf I downloaded yesterday? > What are my top 3 most visited sites? > What type of content do I generally interact with?

That information is already available. You want a better search interface.

Re: MemoryCache: Augmenting local AI with browser data

#25

Was just talking about this on reddit like two days ago Instead of data going to models, we need models come to our data which is stored locally and stay locally. While there are many OSS for Loading personal data, they dont do images or videos. In the future everyone may get their own Model but for now tech is there but product/OSS is missing for everyone to get their own QLORA or RAG or Summarizer. Not just message…

"While there are many OSS for Loading personal data, they dont do images or videos"

Local models for images are getting pretty good.

LLaVA is an LLM with multi-modal image capabilities that runs pretty well on my laptop: https://simonwillison.net/2023/Nov/29/llamafile/

Models like Salesforce BLIP can be used to generate captions for images too - I built a little CLI tool far that here: https://github.com/simonw/blip-caption

Re: MemoryCache: Augmenting local AI with browser data

#26
post #25

Was just talking about this on reddit like two days ago Instead of data going to models, we need models come to our data which is stored locally and stay locally. While there are many OSS for Loading personal data, they dont do images or videos. In the future everyone may get their own Model but for now tech is there but product/OSS is missing for everyone to get their own QLORA or RAG or Summarizer. Not just message…

"While there are many OSS for Loading personal data, they dont do images or videos" Local models for images are getting pretty good. LLaVA is an LLM with multi-modal image capabilities that runs pretty well on my laptop: https://simonwillison.net/2023/Nov/29/llamafile/ Models like Salesforce BLIP can be used to generate captions for images too - I built a little CLI tool far that here: https://github.com/simonw/blip-…

CogVLM blows LLaVA out of the water, although it needs a beefier machine (quantized low-res version barely fits into 12GB VRAM, not sure about the accuracy of that).

Re: MemoryCache: Augmenting local AI with browser data

#27
post #12

What I would love to see is this model being able to learn ti automate some tasks that I usually do! e.g. sign up for events/buy tickets etc! If this has access to your login details, and could log in, it could be a great assistant!

Teach it to press the skip ad button

Re: MemoryCache: Augmenting local AI with browser data

#28

I would sure love a way to "chat" with my browsing history and page content. Is there any way to automatically save off pages that I've visited for later processing? I looked a decade or more ago and didn't really find a good solution.

Rewind.ai is pretty much this - I just installed it and am very happy so far.

Isn't it Apple devices only?

Re: MemoryCache: Augmenting local AI with browser data

#29
post #24

I'm confused by the example they gave. > What is the meaning of a life well-lived? Is the response to this based on browser data? Based on the description I was expecting queries more like: > What was the name of that pdf I downloaded yesterday? > What are my top 3 most visited sites? > What type of content do I generally interact with?

That information is already available. You want a better search interface.

They could private a local url called "about:wrapped" that gives a summary of your usage like Spotify Wrapped. The top 100 sites, you can click on a site for more info like what pages did you visit, when, how often, etc.

Re: MemoryCache: Augmenting local AI with browser data

#30
post #24

I'm confused by the example they gave. > What is the meaning of a life well-lived? Is the response to this based on browser data? Based on the description I was expecting queries more like: > What was the name of that pdf I downloaded yesterday? > What are my top 3 most visited sites? > What type of content do I generally interact with?

That information is already available. You want a better search interface.

Yes, exactly, I want a search interface that's an LLM instead of a bunch of menus.
Post reply on HN