Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Having a 20GB file that lets you ask an offline computer any question is amazing

old.reddit.com

Post reply on HN
Print view View original

61–70 of 127 posts

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#61

Post by eurekin »

d3vmax wrote: post #55
samwillis wrote: post #11

I am completely convinced the future is local LLMs, privet, on your device. But then also fine tuned with your own data. All your documents, email, movements, browser history - maybe it even gets trained on your calls. Effectively an extension to your own brain. I really don't want my AI assistant in the cloud, controlled by a corporation, reading and seeing everything I do. I would go as far as saying I want local A…

I remember google desktop search. It was amazing, even today's win 11 search doesnt come close. So something snappy like that with LLM will be perfect.

Yes! I absolutely loved Google desktop search and hoarded much more pdfs and tex files than otherwise. Very excited for the LLM equivalent.

PrivateGPT, although more of a proof of concept right now, looks like it could be a nice fit in future

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#62

Post by shortcake27 »

jaimex2 wrote: post #54
mFixman wrote: post #52

Earlier quoted context omitted.

Computers are becoming smaller and the internet is becoming even more ubiquous. I don't see your future happening. Why would I spend a lot of money, space, and maintenance time for a computer with a good GPU when I can just use any of the two internet connections I'm already paying for? I'm already trusting corporations to read my email, and no ChatGPT answer can get more personal than that. This is also why I think…

Cloud gaming was a crash and burn story already. There's a stronger resistance to cloud than ever before after all the leaks, hacks and pushed agendas. These models aren't cheap to host and they'll lose to free local models that don't have boundaries or agendas. Cloud AIs inevitably will try and monetise by selling you crap or be funded to shape your opinions.

> Cloud gaming was a crash and burn story already.

What? No it isn’t. One company failing doesn’t mean the industry has crashed and burned. I use Xbox cloud gaming all the time along with millions of other people.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#63

Post by avian »

_neil wrote: post #12
lpuerto wrote: post #6

Why would I do something like that?? Every time I ask something to chatGPT I get wrong answers…

Maybe you’re asking the wrong questions

Correct. Parent commenter seems to be asking questions they already know the answer to.

Those are the wrong kind of questions.

They should only ask the questions about things they don't know anything about.

ChatGPT gets those right every time.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#65

Post by kmac_ »

lastangryman wrote: post #19
noobermin wrote: post #4

The attitude in the comments is amazing. The LLM chatbot hype cycle bears a lot of resemblance to the crypto hype before the crash.

Within a few minutes you can sign up to OpenAI and see the potential of ChatGPT immediately, in you own hands. The potential of crypto/web3 was always an abstract idea (jumped on by so many scammers), there was nothing you could actually use to see the utility.

Plus there is tooling like Copilot that works and is useful. I would love to have something like copilot integrated across my whole system, gathering all my current context, like terminal, notes, mail, issue tracker and have even just plain, simple autocomplete everywhere. It is feasible, and I'm pretty sure we will have more and more integration. I was sceptical, but copilot changed my mind.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#66

Post by stranded22 »

born-jre wrote: post #53
pmontra wrote: post #30

One of the comments on reddit > i have no clue why everyone isn't talking about this all the time. Talking maybe, not doing it because it's still too out of reach for most of us? I've got a laptop with 32 GB RAM and a 2GB NVIDIA card. At which speed can I run that model, if I can run it at all?

2gb vram is not enough for running LLM but you can run on CPU gpt4all provides gui just click click download model click and you are ready, just give it a try start with `mpt-chat7b`, my suggestion https://gpt4all.io/index.html

Thank you for that, I’m installing now

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#67

Post by npteljes »

karmakurtisaani wrote: post #51
dharmab wrote: post #34

Earlier quoted context omitted.

At the risk of conflict of interest, there's a wikipedia page that acknowledges that niche sections of the site have biases. Of particular interest is the section on Croatia. https://en.m.wikipedia.org/wiki/Ideological_bias_on_Wikipedi...

It baffles me how you can either have a news/information source with a left-wing bias or a total cesspool of untrustworthy information with a glaringly obvious agenda. This comment seems to get some hate, but all you have to do is skim over the linked Wikipedia article to verify what I'm saying.

I agree, and for another demonstration, I invite everyone to look at similar sites as Wikipedia. They are either "left wing bias" (RationalWiki) or one with an agenda, divorced from reality (Conservapedia). It almost looks like that if you organize basic democratic cooperation among people, that's automatically "left wing" from where our current narrative stands.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#68

Post by [deleted] »

midasuni wrote: post #28
kramerger wrote: post #8

We can do that, but then we will need to close all applications to run it because it will need 20GB RAM or VRAM to run. It is still more practical to visit stackoverflow when I have a question. (still impressive how much this improves every week. Maybe in 2-3 months we have something more useful)

I just upgraded my 2020 era thinkpad (t470, which I bought this time last year for less than £300) to 32g memory for £70. 20g of ram doesn’t feel like a lot today.

[deleted]

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#69

Post by tinus_hn »

samwillis wrote: post #38
sircastor wrote: post #22

Earlier quoted context omitted.

I also don’t want my AI in the cloud favoring a corporation’s values and goals. But these tools are almost certainly going to come from them. And they want you to store all your data in the cloud too. And they want to keep you from running the software you want.

I think there is one company very well placed to do this, Apple. Their revenue is driven by hardware sales, rather than advertising, and they have committed to some level of "privacy" focussed design (although obviously there are criticisms). On top of that their hardware is looking almost like it was design for exactly these use cases. Apple silicone with its unified memory architecture, GPU and Neural cores is very…

I think Apple will be hesitant to release an AI on their platforms that can be made to say unpredictably bad things. For now as far as I have seen there’s only censoring and that’ll only lead to a game of people trying to work around it.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#70

Post by thinkingemote »

mnd999 wrote: post #39
petesergeant wrote: post #16

Earlier quoted context omitted.

I would pay several hundred dollars a month to keep access to GPT-4, and never found any use at all for crypto. There might be hype, but there’s also an insane amount of utility

Some examples please. I’ve yet to find any actual utility, mostly I’ve got a lot of convincing bullshit out of it.

For me it's not the answer but the formatting or presentation of it.

So it's like a normal search but where you don't need to reformat and rearrange the result to suit your needs.

Post reply on HN