Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MemoryCache: Augmenting local AI with browser data

future.mozilla.org

Post reply on HN
Print view View original

101–108 of 108 posts

Re: MemoryCache: Augmenting local AI with browser data

#101

Post by bjord »

conradev wrote: post #63
Jayakumark wrote: post #2

Was just talking about this on reddit like two days ago Instead of data going to models, we need models come to our data which is stored locally and stay locally. While there are many OSS for Loading personal data, they dont do images or videos. In the future everyone may get their own Model but for now tech is there but product/OSS is missing for everyone to get their own QLORA or RAG or Summarizer. Not just message…

> Instead of data going to models, we need models come to our data which is stored locally and stay locally. We are building this over at https://software.inc ! We collect data about you (from your computer and the internet) into a local database and then teach models how to use it. The models can either be local or cloud-based, and we can route requests based on the sensitivity of the data or the capabilities needed…

your site is not loading at all for me on firefox (emulator error) and is totally non-functional on chrome (TCPCreate failed)

might be worth having some sort of automatic fallback to a static site after a certain amount of failed loading or an error

just saw the link to your html version in another comment and it took literally five minutes to load on firefox

Re: MemoryCache: Augmenting local AI with browser data

#102

Post by emsign »

Does it ingest the ads containing in the web pages as well? This would be a major concern, ads will pollute the model at least with unwanted information. At worst they'd be a security concern when used for indiscriminate or even targeted manipulation of the model. Advertisers do that to our brains, it's all they do. So why shouldn't they try that with LLMs scraping/being fed from the web?

Re: MemoryCache: Augmenting local AI with browser data

#103

Post by jval43 »

ayewo wrote: post #100
jval43 wrote: post #52

Earlier quoted context omitted.

Didn't Chrome do this at the very beginning, when it was initially released? I faintly remember that being a feature. Personally I would already be content if my browsers didn't forget their history all the time, both Firefox and Safari history is way too short-lived.

You were probably thinking of Google Desktop which could search almost anything on your machine. https://en.wikipedia.org/wiki/Google_Desktop

I went looking and it was indeed Chrome that could do it. See screenshot from 2009 here: https://superuser.com/a/42499

Google removed the feature intentionally in 2013: https://bugs.chromium.org/p/chromium/issues/detail?id=297648

Apparently Opera supported it too at the time, and from the comments Safari as well.

Performance reasons seem to have killed it. I'd think that after 10 years now any modern computer would be able to handle it.

Re: MemoryCache: Augmenting local AI with browser data

#104

Post by dr_kiszonka »

unstuck3958 wrote: post #98
dr_kiszonka wrote: post #94

Earlier quoted context omitted.

Do you know if there is a website or spreadsheet where I could enter my RAM, GPU, etc. and see what models I could run locally?

While I agree a website/spreadsheet would be convenient, it's not that complicated. As long as GPU is handling 50-75% of the LLM layers, you should get a decent tok/sec speed (unless you're running really really large models).

Could you explain to me (in steps) how I would go about calculating how much VRAM I would need to run, say, Mistral 8x7B?

Re: MemoryCache: Augmenting local AI with browser data

#105

Post by edsu »

avallach wrote: post #5

PrivateGPT repository in case anyone's interested: https://github.com/imartinez/privateGPT . It doesn't seem to be linked from their official website.

Does anyone know what PrivateGPT is using for its local model, and where it came from?

Update:

Answering my own question it looks like it uses llamacpp in local mode? https://github.com/imartinez/privateGPT/blob/main/private_gp...

Re: MemoryCache: Augmenting local AI with browser data

#106

Post by edsu »

nekitamo wrote: post #22
eigenvalue wrote: post #20

Earlier quoted context omitted.

Agree, it seems like it’s insanely hard to back up a modern JS-enabled web page in a usable way that results in a single file which can be easily shared.

Have you tried SingleFile? It sounds like what you’re looking for: https://github.com/gildas-lormeau/SingleFile

Also check out https://archiveweb.page which is open source, local, and lets you export archived data as WARC (ISO 28500). You can embed archives in web pages using their Web Component https://replayweb.page.

Re: MemoryCache: Augmenting local AI with browser data

#107

Post by edsu »

Sai_ wrote: post #46
Dwedit wrote: post #9

Very misleading name. The word "Memory" has a distinct meaning in relation to computing, but this is more about human memories.

I was going to ignore this as a troll comment because computer memory has its antecedents in human memory but the commenter is right - the combination of memory and cache to talk about human memory seems misleading.

I kind of like the association since it speaks to how text collected while browsing the web can be used to generate new text, which is similar, at least metaphorically, to how human memory is reconstructive and transformative, not perfect recall. https://en.wikipedia.org/wiki/Reconstructive_memory

Re: MemoryCache: Augmenting local AI with browser data

#108

Post by ayewo »

jval43 wrote: post #103
ayewo wrote: post #100

Earlier quoted context omitted.

You were probably thinking of Google Desktop which could search almost anything on your machine. https://en.wikipedia.org/wiki/Google_Desktop

I went looking and it was indeed Chrome that could do it. See screenshot from 2009 here: https://superuser.com/a/42499 Google removed the feature intentionally in 2013: https://bugs.chromium.org/p/chromium/issues/detail?id=297648 Apparently Opera supported it too at the time, and from the comments Safari as well. Performance reasons seem to have killed it. I'd think that after 10 years now any modern computer would b…

That's an interesting find.
Post reply on HN