Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Having a 20GB file that lets you ask an offline computer any question is amazing

old.reddit.com

Post reply on HN
Print view View original

111–120 of 127 posts

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#111

Post by ProllyInfamous »

_xnmw wrote: post #59

Now I have an excuse to purchase high powered machines. As a programmer who doesn't play games, edit video, or do anything requiring a high-powered graphics card, I always eyed high-end rigs as a wasteful indulgence. Not anymore! Now I can't wait to train my own models on my own data and make the computer do things for me, like its 1982 all over again.

I recently purchased my first new computer since 2008 — a Mac Mini M2Pro, stock.

It's incredible what this $1300 machine can do, almost out of the box (e.g. DiffusionBee).

From my understanding the device shares RAM/VRAM, and has a total of 16GB (up to 32), 14.5~GB of which can be used for LLM/genAI processess.

I am just a layperson, a non-gamer; and this purchase has been really wild for local generations.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#112

Post by schoen »

cassianoleal wrote: post #87
schoen wrote: post #2

I was pretty impressed by the power and accuracy of the Google Translate offline models (which predate some of the amazing new LLMs), also in a seemingly amazingly small space—tens of megabytes per language, if I remember correctly?

Firefox's builtin translation is not bad either. I've seen it make a mistake here and there, or fail to translate certain words and expressions but it works generally well.

Is it an offline system, not using an API?

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#113

Post by cassianoleal »

schoen wrote: post #112
cassianoleal wrote: post #87

Earlier quoted context omitted.

Firefox's builtin translation is not bad either. I've seen it make a mistake here and there, or fail to translate certain words and expressions but it works generally well.

Is it an offline system, not using an API?

Yes.

"The Firefox Translations extension can automatically translate content from the web pages you visit. Unlike some cloud-based alternatives, this extension translates text locally in Firefox, so the content you’re translating doesn’t leave your machine."

https://addons.mozilla.org/en-GB/firefox/addon/firefox-trans...

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#114

Post by tatrajim »

cookieperson wrote: post #90
bheadmaster wrote: post #37

Earlier quoted context omitted.

US politics, definitely. But also any other topic that attracts an internet mob large enough to infiltrate the editor community. Take the first few paragraphs of Donald Trump [0] vs. Joe Biden [1] and try to compare the way they're presented: Donald John Trump [...] and his businesses have been involved in more than 4,000 state and federal legal actions, including six business bankruptcies. Trump promoted conspiracy…

This is actually a pretty sound summary of the two. The one guy just got charged for rape, recently another one of his companies went down for fraud, he invited a riot on j6 to overthrow democracy, is under investigation for stealing hundreds of classified documents, lied tens of thousands of times(people take the time to count these things with references for presidents), issued dozens of legal challenges to try to…

Award for best understated comedy on this thread.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#115

Post by npteljes »

zirgs wrote: post #80
npteljes wrote: post #67

Earlier quoted context omitted.

I agree, and for another demonstration, I invite everyone to look at similar sites as Wikipedia. They are either "left wing bias" (RationalWiki) or one with an agenda, divorced from reality (Conservapedia). It almost looks like that if you organize basic democratic cooperation among people, that's automatically "left wing" from where our current narrative stands.

Some parts of RationalWiki aren't that rational any more, unfortunately.

Eh, if most is rational, then I think they do a good job. Very factual too, which is good to know:

https://mediabiasfactcheck.com/rationalwiki/

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#116

Post by schoen »

cassianoleal wrote: post #113
schoen wrote: post #112

Earlier quoted context omitted.

Is it an offline system, not using an API?

Yes. "The Firefox Translations extension can automatically translate content from the web pages you visit. Unlike some cloud-based alternatives, this extension translates text locally in Firefox, so the content you’re translating doesn’t leave your machine." https://addons.mozilla.org/en-GB/firefox/addon/firefox-trans...

Cool, I'll give that a try! I didn't know about it before.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#117

Post by m463 »

avian wrote: post #63
_neil wrote: post #12

Earlier quoted context omitted.

Maybe you’re asking the wrong questions

Correct. Parent commenter seems to be asking questions they already know the answer to. Those are the wrong kind of questions. They should only ask the questions about things they don't know anything about. ChatGPT gets those right every time.

fun laws - the Gell-Mann amnesia effect / Knoll's law of media accuracy:

https://en.wikipedia.org/wiki/Michael_Crichton#GellMannAmnes...

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#118

Post by jaimex2 »

triggerity wrote: post #76
jaimex2 wrote: post #54

Earlier quoted context omitted.

Cloud gaming was a crash and burn story already. There's a stronger resistance to cloud than ever before after all the leaks, hacks and pushed agendas. These models aren't cheap to host and they'll lose to free local models that don't have boundaries or agendas. Cloud AIs inevitably will try and monetise by selling you crap or be funded to shape your opinions.

Gaming is a total different field. Hard to make any assumptions out of this.most people I know definitely sync there data onto companies cloud storage. Alone for image backups and sharing and etc. No one minds that. No one minds mail analysis either. Cloud is healthy, very healthy

It is, but AI is a different kettle of fish altogether.

We already push back on targeted advertising and tracking, think what AI is capable of with your data. If its writing your emails, if its summarising for you, if its consulting for you...

It can push agendas and influence. Its quite dangerous.

These models need to be local as much as possible and they need to be better than what the cloud can offer.

At least for me this isnt one the decentralised Internet can take if we don't want to go down the dystopian future route.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#119

Post by fomine3 »

jaimex2 wrote: post #54
mFixman wrote: post #52

Earlier quoted context omitted.

Computers are becoming smaller and the internet is becoming even more ubiquous. I don't see your future happening. Why would I spend a lot of money, space, and maintenance time for a computer with a good GPU when I can just use any of the two internet connections I'm already paying for? I'm already trusting corporations to read my email, and no ChatGPT answer can get more personal than that. This is also why I think…

Cloud gaming was a crash and burn story already. There's a stronger resistance to cloud than ever before after all the leaks, hacks and pushed agendas. These models aren't cheap to host and they'll lose to free local models that don't have boundaries or agendas. Cloud AIs inevitably will try and monetise by selling you crap or be funded to shape your opinions.

> There's a stronger resistance to cloud than ever before after all the leaks, hacks and pushed agendas.

It's a thing on HN loop.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#120

Post by cookieperson »

roarcher wrote: post #109
cookieperson wrote: post #89

Earlier quoted context omitted.

So a hackers main goal will be to get your LLM data, and companies will be backdooring them?

Why would this be a risk? The data is already on your machine. It would be much easier for a hacker to target that data directly, as they do today, than to try to extract some impossibly obfuscated form of it from an LLM.

Well if they are interested in data. If they are interested in asking a prompt "hey what's my cc number and password again?" That'd bypass an awful lot of encryption. Would also be easy to find blackmail for a lot of people thinking it's a secure way to lookup crimes.
Post reply on HN