Live data from Hacker News

Having a 20GB file that lets you ask an offline computer any question is amazing

old.reddit.com

31–40 of 127 posts

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#31

I am completely convinced the future is local LLMs, privet, on your device. But then also fine tuned with your own data. All your documents, email, movements, browser history - maybe it even gets trained on your calls. Effectively an extension to your own brain. I really don't want my AI assistant in the cloud, controlled by a corporation, reading and seeing everything I do. I would go as far as saying I want local A…

https://mitta.us is one project that I find myself attracted to but haven't dare to try because it isn't local.

It does 25% of what you ask (subtracted 50% because it isn't local and 25% because it is missing calls, movies etc).

I wonder if at some point there will be an open source project like this?

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#33
post #5

You can download a local version of Wikipedia and yet barely anyone does this. (And I know wikipedia has a ton of problems)

Yeah it's a pretty fun dataset that you can do a lot of stuff with. Very slept on.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#34
post #25

Earlier quoted context omitted.

Given the biased nature of many if not most Wikipedia articles it would be certain that such a system would bend the truth to an unacceptable degree. Train the thing on more than just Wikipedia, e.g add Encyclopedia Britannica to the training set. This is the one area where "diversity" really matters: diversity of opinion. Using a single biased source as your oracle will turn you into a pawn for those who control the…

what? since when wikipedia is biased? and about what? (and why EB isn't?)

At the risk of conflict of interest, there's a wikipedia page that acknowledges that niche sections of the site have biases. Of particular interest is the section on Croatia.

https://en.m.wikipedia.org/wiki/Ideological_bias_on_Wikipedi...

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#35
post #30

One of the comments on reddit > i have no clue why everyone isn't talking about this all the time. Talking maybe, not doing it because it's still too out of reach for most of us? I've got a laptop with 32 GB RAM and a 2GB NVIDIA card. At which speed can I run that model, if I can run it at all?

2GB VRAM means you can run things comparable to GPT2, a glorified Markov chain. On your CPU you could run much larger models at far from real time speeds.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#36
post #6

Why would I do something like that?? Every time I ask something to chatGPT I get wrong answers…

I don't know about every time but honestly the fact that it'll just fill in the gaps when it doesn't know means that you have to treat all responses with a bit of suspicion.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#37
post #25

Earlier quoted context omitted.

Given the biased nature of many if not most Wikipedia articles it would be certain that such a system would bend the truth to an unacceptable degree. Train the thing on more than just Wikipedia, e.g add Encyclopedia Britannica to the training set. This is the one area where "diversity" really matters: diversity of opinion. Using a single biased source as your oracle will turn you into a pawn for those who control the…

what? since when wikipedia is biased? and about what? (and why EB isn't?)

US politics, definitely. But also any other topic that attracts an internet mob large enough to infiltrate the editor community.

Take the first few paragraphs of Donald Trump [0] vs. Joe Biden [1] and try to compare the way they're presented:

    Donald John Trump [...] and his businesses have been involved in more than 4,000 state and federal legal actions, including six business bankruptcies. Trump promoted conspiracy theories and made many false and misleading statements during his campaigns and presidency, to a degree unprecedented in American politics. Many of his comments and actions have been characterized as racially charged or racist and many as misogynistic. [...] He reacted slowly to the COVID-19 pandemic, ignored or contradicted many recommendations from health officials, used political pressure to interfere with testing efforts, and spread misinformation about unproven treatments. 
There's not a single positive sentence about him. He must have done something positive in his 4 years of reign, but Wikipedia doesn't tell me that in an objective manner. Meanwhile...

    Joseph Robinette Biden Jr. [...] has addressed the COVID-19 pandemic and subsequent recession. He signed the American Rescue Plan Act, the bipartisan Infrastructure Investment and Jobs Act, the Inflation Reduction Act, the CHIPS and Science Act, and the Respect for Marriage Act, which codified protections for same-sex marriage and repealed the Defense of Marriage Act. He appointed Ketanji Brown Jackson to the Supreme Court. In foreign policy, Biden restored America's membership in the Paris Agreement on climate change.
There's not a single negative sentence about him. He must have done something negative in his 4 years of reign, but Wikipedia doesn't tell me that in an objective manner.

Whenever someone tries to portray someone as completely evil or completely saint-like, I call bullshit.

[0] https://en.wikipedia.org/wiki/Donald_Trump

[1] https://en.wikipedia.org/wiki/Joe_Biden

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#38

I am completely convinced the future is local LLMs, privet, on your device. But then also fine tuned with your own data. All your documents, email, movements, browser history - maybe it even gets trained on your calls. Effectively an extension to your own brain. I really don't want my AI assistant in the cloud, controlled by a corporation, reading and seeing everything I do. I would go as far as saying I want local A…

I also don’t want my AI in the cloud favoring a corporation’s values and goals. But these tools are almost certainly going to come from them. And they want you to store all your data in the cloud too. And they want to keep you from running the software you want.

I think there is one company very well placed to do this, Apple. Their revenue is driven by hardware sales, rather than advertising, and they have committed to some level of "privacy" focussed design (although obviously there are criticisms). On top of that their hardware is looking almost like it was design for exactly these use cases. Apple silicone with its unified memory architecture, GPU and Neural cores is very well placed for local LLMs.

Apple have a history of doing stuff locally, on the hardware they designed, that other companies have offloaded to the cloud.

They are looking for "the next big thing" that lets them sell hardware. Some people thinks that's going to be their VR/AR headset, I don't. It's going to be local AI running privately on your device. Putting the AI in the cloud doesn't help with hardware sales.

My only hesitation is that they have such a crap spell checker, surly you would solve first...

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#39

The attitude in the comments is amazing. The LLM chatbot hype cycle bears a lot of resemblance to the crypto hype before the crash.

I would pay several hundred dollars a month to keep access to GPT-4, and never found any use at all for crypto. There might be hype, but there’s also an insane amount of utility

Some examples please. I’ve yet to find any actual utility, mostly I’ve got a lot of convincing bullshit out of it.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#40
post #25

Earlier quoted context omitted.

Given the biased nature of many if not most Wikipedia articles it would be certain that such a system would bend the truth to an unacceptable degree. Train the thing on more than just Wikipedia, e.g add Encyclopedia Britannica to the training set. This is the one area where "diversity" really matters: diversity of opinion. Using a single biased source as your oracle will turn you into a pawn for those who control the…

what? since when wikipedia is biased? and about what? (and why EB isn't?)

Really? Wikipedia's left-wing bias is well known:

https://www.foxnews.com/politics/wikipedia-bias-socialism-pa...

https://twitter.com/elonmusk/status/1600200000763756544

https://www.dailymail.co.uk/news/article-9283061/Wikipedia-f...

https://nypost.com/2021/07/16/wikipedia-co-founder-says-site...

https://www.dailysignal.com/2022/04/27/wikipedias-left-wing-...

Compared to Britannica:

https://hbswk.hbs.edu/item/is-wikipedia-more-biased-than-enc...

Post reply on HN