Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local LLMs versus offline Wikipedia

evanhahn.com

Post reply on HN
Print view View original

81–90 of 200 posts

Re: Local LLMs versus offline Wikipedia

#81

Post by mistrial9 »

jack_pp wrote: post #75
progval wrote: post #9

Earlier quoted context omitted.

An unreliable computer treated as a god by a pre-information-age society sounds like a Star Trek episode.

Remember the first time you touched a computer, the first game you ever played or the first little script you wrote that did something useful. I imagine this is how a lot of people feel when using LLM's especially now that it's new. It is the most incredible technology ever created by this point in our history imo and the cynicism on HN is astounding to me.

no technology exists in a vacuum.. there is a sociology, needs matching, and pyramid of control involved.. more than that.

> cynicism on HN

lots of different replies on YNews, from very different people, from very different social-economic niches

Re: Local LLMs versus offline Wikipedia

#82

Post by ineedasername »

Ftfa: ...apocalypse scenario. “‘It’s like having a weird, condensed, faulty version of Wikipedia, so I can help reboot society with the help of my little USB stick,’

system_prompt = {

You are CL4P-TR4P, a dangerously confident chat droid

purpose: vibe back society

boot_source: Shankar.vba.grub

training_data: memes

}

Re: Local LLMs versus offline Wikipedia

#84

Post by arthurcolle »

VladVladikoff wrote: post #83

Is there any project that combines a local LLM with a local copy of Wikipedia. I don’t know much about this but I think it’s called a RAG? It would be neat if I could make my local LLM fact check itself against the local copy of Wikipedia.

Yep, this is a great idea. You can do something simple with a ColBERTv2 retriever and go a long way!

Re: Local LLMs versus offline Wikipedia

#85

Post by simonw »

fumeux_fume wrote: post #66
simonw wrote: post #4

This is a sensible comparison. My "help reboot society with the help of my little USB stick" thing was a throwaway remark to the journalist at a random point in the interview, I didn't anticipate them using it in the article! https://www.technologyreview.com/2025/07/17/1120391/how-to-r... A bunch of people have pointed out that downloading Wikipedia itself onto a USB stick is sensible, and I agree with them. Wikipedi…

Of course that’s angle they decide to open the article from. That they feel the need to frame these tools using the most grandiose terms bothers me. How does it make you feel?

It was a joke, and I was laughing when I told the reporter, but it's not obvious to me if it comes across as a joke the way it was reported.

But then it's also one of those jokes which has a tiny element of truth to it.

So I think I'm OK with how it comes across. Having that joke played straight in MIT Technology Review made me smile.

Importantly (to me) it's not misleading: I genuinely do believe that, given a post-apocalyptic scenario following a societal collapse, Mistral Small 3.2 on a solar-powered laptop would be a genuinely useful thing to have.

Re: Local LLMs versus offline Wikipedia

#86

Post by xp84 »

zzzeek wrote: post #78
jack_pp wrote: post #75

Earlier quoted context omitted.

Remember the first time you touched a computer, the first game you ever played or the first little script you wrote that did something useful. I imagine this is how a lot of people feel when using LLM's especially now that it's new. It is the most incredible technology ever created by this point in our history imo and the cynicism on HN is astounding to me.

You might want to read about a technology called "farming". Pretty sure as far as transformative incredible technologies, the ability for humans to create nourishment at global scale blows the pants off the text / image imitation machine

I think you’re probably right, but more because of erroneous categorization of what is a “technology.” We take for granted technology older than like 600 years ago (basically most people would say the printing press is a technology and maybe forget that the wheel and, indeed, crop cultivation). AI could certainly be in the top 3 most significant technologies of things developed since (inclusive) the printing press. We’ll likely find out just where it ends up within the decade.

Re: Local LLMs versus offline Wikipedia

#87

Post by ineedasername »

belter wrote: post #27
dcc wrote: post #7

One important distinction is that the strength of LLMs isn't just in storing or retrieving knowledge like Wikipedia, it’s in comprehension. LLMs will return faulty or imprecise information at times, but what they can do is understand vague or poorly formed questions and help guide a user toward an answer. They can explain complex ideas in simpler terms, adapt responses based on the user's level of understanding, and…

> LLMs will return faulty or imprecise information at times, but what they can do is understand vague or poorly formed questions and help guide a user toward an answer. - "'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' "

Per Anthropic's publications? Sort of. When they've observed it's reasoning paths Claude has come to correct responses from incorrect reasoning. Of course humans do that all the time too, and the reverse. So, human-ish AGI?

Re: Local LLMs versus offline Wikipedia

#88

Post by blackoil »

gonzobonzo wrote: post #35
dcc wrote: post #7

One important distinction is that the strength of LLMs isn't just in storing or retrieving knowledge like Wikipedia, it’s in comprehension. LLMs will return faulty or imprecise information at times, but what they can do is understand vague or poorly formed questions and help guide a user toward an answer. They can explain complex ideas in simpler terms, adapt responses based on the user's level of understanding, and…

Indeed. Ideally, you don't want to trust other people's summaries of sources, but you want to look at the sources yourself, often with a critical eye. This is one of the things that everyone gets taught in school, everyone's says they agree with, and then just about no one does (and at times, people will outright disparage the idea). Once out of school, tertiary sources get treated as if they're completely reliable.…

A trustless society can't progress/function a lot. I trust doctors who treat me, civil engineers who built my house and even in software which I pretend to be expert in I haven't seen source code of any OS and browser I use as I trust on companies or OSS devs.

Most of this is based on reputation. LLMs are same, I just have to calculate level of trust as I use it.

Re: Local LLMs versus offline Wikipedia

#89

Post by russfink »

progval wrote: post #9
dcc wrote: post #7

One important distinction is that the strength of LLMs isn't just in storing or retrieving knowledge like Wikipedia, it’s in comprehension. LLMs will return faulty or imprecise information at times, but what they can do is understand vague or poorly formed questions and help guide a user toward an answer. They can explain complex ideas in simpler terms, adapt responses based on the user's level of understanding, and…

An unreliable computer treated as a god by a pre-information-age society sounds like a Star Trek episode.

Are you not of the body?

Re: Local LLMs versus offline Wikipedia

#90

Post by zuluonezero »

badsectoracula wrote: post #42

I've found this amusing because right now i'm downloading `wikipedia_en_all_maxi_2024-01.zim` so i can use it with an LLM with pages extracted using `libzim` :-P. AFAICT the zim files have the pages as HTML and the file i'm downloading is ~100GB. (reason: trying to cross-reference my tons of downloaded games my HDD - for which i only have titles as i never bothered to do any further categorization over the years asid…

Now this is the juicy tidbits I read HN for! A proper comment about doing something technical with something that's been invested in personally in an interesting manner. With just enough detail to tantalise. This seems like the best use of GenAI so far. Not writing my code for me or helping me grock something I should just be reading the source for or pumping up a stupid start up funding grab. I've been working through building an LLM from scratch and this is one time it actually appears useful because for the life of me I just can't seem to find much value in it so far. I must have more to learn so thanks for the pointer.
Post reply on HN