Live data from Hacker News

Having a 20GB file that lets you ask an offline computer any question is amazing

old.reddit.com

51–60 of 127 posts

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#51
post #34
post #25

Earlier quoted context omitted.

what? since when wikipedia is biased? and about what? (and why EB isn't?)

At the risk of conflict of interest, there's a wikipedia page that acknowledges that niche sections of the site have biases. Of particular interest is the section on Croatia. https://en.m.wikipedia.org/wiki/Ideological_bias_on_Wikipedi...

It baffles me how you can either have a news/information source with a left-wing bias or a total cesspool of untrustworthy information with a glaringly obvious agenda.

This comment seems to get some hate, but all you have to do is skim over the linked Wikipedia article to verify what I'm saying.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#52

I am completely convinced the future is local LLMs, privet, on your device. But then also fine tuned with your own data. All your documents, email, movements, browser history - maybe it even gets trained on your calls. Effectively an extension to your own brain. I really don't want my AI assistant in the cloud, controlled by a corporation, reading and seeing everything I do. I would go as far as saying I want local A…

Computers are becoming smaller and the internet is becoming even more ubiquous. I don't see your future happening.

Why would I spend a lot of money, space, and maintenance time for a computer with a good GPU when I can just use any of the two internet connections I'm already paying for? I'm already trusting corporations to read my email, and no ChatGPT answer can get more personal than that.

This is also why I think cloud gaming is only going to become more popular.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#53
post #30

One of the comments on reddit > i have no clue why everyone isn't talking about this all the time. Talking maybe, not doing it because it's still too out of reach for most of us? I've got a laptop with 32 GB RAM and a 2GB NVIDIA card. At which speed can I run that model, if I can run it at all?

2gb vram is not enough for running LLM but you can run on CPU gpt4all provides gui just click click download model click and you are ready, just give it a try

start with `mpt-chat7b`, my suggestion

https://gpt4all.io/index.html

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#54
post #52

I am completely convinced the future is local LLMs, privet, on your device. But then also fine tuned with your own data. All your documents, email, movements, browser history - maybe it even gets trained on your calls. Effectively an extension to your own brain. I really don't want my AI assistant in the cloud, controlled by a corporation, reading and seeing everything I do. I would go as far as saying I want local A…

Computers are becoming smaller and the internet is becoming even more ubiquous. I don't see your future happening. Why would I spend a lot of money, space, and maintenance time for a computer with a good GPU when I can just use any of the two internet connections I'm already paying for? I'm already trusting corporations to read my email, and no ChatGPT answer can get more personal than that. This is also why I think…

Cloud gaming was a crash and burn story already.

There's a stronger resistance to cloud than ever before after all the leaks, hacks and pushed agendas.

These models aren't cheap to host and they'll lose to free local models that don't have boundaries or agendas. Cloud AIs inevitably will try and monetise by selling you crap or be funded to shape your opinions.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#55

I am completely convinced the future is local LLMs, privet, on your device. But then also fine tuned with your own data. All your documents, email, movements, browser history - maybe it even gets trained on your calls. Effectively an extension to your own brain. I really don't want my AI assistant in the cloud, controlled by a corporation, reading and seeing everything I do. I would go as far as saying I want local A…

I remember google desktop search. It was amazing, even today's win 11 search doesnt come close. So something snappy like that with LLM will be perfect.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#56

Learning is going to be revolutionized.

Yes, this. Once they get a smidge better and can answer questions? Every student will have the King's Tutor - it will be Blooms Two Sigma in action.

...which will drive down the cost of all knowledge work and rather than reconfigure our society we'll simply have no entry level jobs anymore and the few people who still have jobs will still be expected to work full time but have insane outputs.

This whole thing is a Monkey's Paw.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#57
post #39

Earlier quoted context omitted.

I would pay several hundred dollars a month to keep access to GPT-4, and never found any use at all for crypto. There might be hype, but there’s also an insane amount of utility

Some examples please. I’ve yet to find any actual utility, mostly I’ve got a lot of convincing bullshit out of it.

For almost anything, I find it a better Google, and I can go fact-check it afterwards.

It’s a better StackOverflow and better than most project or dev docs for me: I’m an experienced developer who’s writing Python for the first time. I know what I want to achieve each time (how do I produce the Typing type for…) but I find it much easier to use GPT.

I use it to search for books or films or video games where I half remember the details, and it gives me options.

I recently finished a dissertation (on GPT, as it turns out) and it was useful for me to research sensible correlation types I could use and build out my understanding (“wait, is Spearman’s rank is just Pearson in the rank order?”). Also I could copy and paste sentences I didn’t understand about methodologies in papers I was reading, or ask it abstract questions like “is there a standard name for a modified Likert scale but with a ‘not enough data’ option”.

I was really struggling to work out how to change just one page orientation in Word, on a Mac. First few Google results I had talked about the Windows version, which had features mine didn’t have. GPT put me onto section breaks.

Was Caffè Nero named after the emperor, checking my intuition about how short-term SSH auth certs work, jq recipes, why were Man U so popular when I was a kid, why did they suddenly start winning more often?

I am curious about a lot of things, and GPT provides me instant answers without having to click through bullshit. If it’s something it needs to be right about I can double-check it’s answers. I am averaging ten topics a day on it, most have a bunch of follow-up questions.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#58

Earlier quoted context omitted.

US politics, definitely. But also any other topic that attracts an internet mob large enough to infiltrate the editor community. Take the first few paragraphs of Donald Trump [0] vs. Joe Biden [1] and try to compare the way they're presented: Donald John Trump [...] and his businesses have been involved in more than 4,000 state and federal legal actions, including six business bankruptcies. Trump promoted conspiracy…

The article covers allegations of corruption and mentions his "gaffes" a number of times. Sometimes where there's no smoke, there's no fire. The guy is very boring and has a weird kid. That's about it.

i know you're being sarcastic, but do i think its interesting that centrist politics is sometimes viewed as less corrupt just because the bias is harder to articulate

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#59
Now I have an excuse to purchase high powered machines. As a programmer who doesn't play games, edit video, or do anything requiring a high-powered graphics card, I always eyed high-end rigs as a wasteful indulgence. Not anymore! Now I can't wait to train my own models on my own data and make the computer do things for me, like its 1982 all over again.

Re: Having a 20GB file that lets you ask an offline computer any question is amazing

#60
post #25

Earlier quoted context omitted.

what? since when wikipedia is biased? and about what? (and why EB isn't?)

US politics, definitely. But also any other topic that attracts an internet mob large enough to infiltrate the editor community. Take the first few paragraphs of Donald Trump [0] vs. Joe Biden [1] and try to compare the way they're presented: Donald John Trump [...] and his businesses have been involved in more than 4,000 state and federal legal actions, including six business bankruptcies. Trump promoted conspiracy…

Your [...] are very convenient, because you skipped over the part in paragraph 3 of Trump's article where they go over some of his main "achievements", which are presented neutrally and some people might consider positive:

    [Trump] signed the Tax Cuts and Jobs Act of 2017, which cut taxes for individuals and businesses and rescinded the individual health insurance mandate penalty of the Affordable Care Act. He appointed 54 federal appellate judges and three U.S. Supreme Court justices. He initiated a trade war with China and withdrew the U.S. from the proposed Trans-Pacific Partnership trade agreement, the Paris Agreement on climate change, and the Iran nuclear deal.
and you also cut out a mention of a pretty significant Biden failure (in outcome, at least):

    [Biden] oversaw the withdrawal of U.S. troops from Afghanistan that had been negotiated and begun under the previous administration, ending the war in Afghanistan, during which the Afghan government collapsed and the Taliban seized control.
But moreover, the introductory paragraphs of a Wikipedia article, like the introduction to most pieces of writing, are meant to establish an overview. Assuming that the reader reads nothing else, what basic idea should they have of the thing?

Donald Trump spent his four years in office much like he spent most of his previous business career: lying and committing various forms of fraud. While president, just for variety, he added lying and committing fraud about COVID, and, oh yes, inciting a violent insurrection to overthrow the American government. That should be the first thing you take away. He may have accidentally done some positive things, but none of them make the lead paragraph, because they won't be in the top 3-5 things that history remembers him for. "Scholars and historians rank Trump as one of the worst presidents in American history" appears near the end of the introduction (right before jumping back into more recent events), because that's a germane summary.

Joe Biden has spent his 2+ years in office being a relatively uninteresting mainstream-to-slightly-progressive Democrat. That should be the first thing you take away. He's not perfect, but he's doing normal president stuff. Once he's no longer president, no doubt his article will be updated to look like Obama's, which currently ends the introduction with:

    Rankings by scholars and historians, in which [Obama] has been featured since 2010, place him in the middle to upper tier of American presidents.
...that is, not saintlike, but, you know, fine.
Post reply on HN