I currently have a copy of offline Wikipedia (and Stack Overflow, WikiVoyage, etc.) via Kiwix on my laptop and also LLaMA-style models at the same time (since I like things being local), and with respect to asking an LLM questions, it would be great if it could leverage the offline Wikipedias to ensure it stays grounded and not make up facts.
Having a 20GB file that lets you ask an offline computer any question is amazing
21–30 of 127 posts
Re: Having a 20GB file that lets you ask an offline computer any question is amazing
#22I am completely convinced the future is local LLMs, privet, on your device. But then also fine tuned with your own data. All your documents, email, movements, browser history - maybe it even gets trained on your calls. Effectively an extension to your own brain. I really don't want my AI assistant in the cloud, controlled by a corporation, reading and seeing everything I do. I would go as far as saying I want local A…
I also don’t want my AI in the cloud favoring a corporation’s values and goals.
But these tools are almost certainly going to come from them. And they want you to store all your data in the cloud too. And they want to keep you from running the software you want.
Re: Having a 20GB file that lets you ask an offline computer any question is amazing
#23The attitude in the comments is amazing. The LLM chatbot hype cycle bears a lot of resemblance to the crypto hype before the crash.
Though there are people selling pickaxes at gold rush, it's not pump and dump scheme like most crypto
Re: Having a 20GB file that lets you ask an offline computer any question is amazing
#24The attitude in the comments is amazing. The LLM chatbot hype cycle bears a lot of resemblance to the crypto hype before the crash.
LLM has provided real value for me the last couple of months, both as amusement, as a way to "talk" while working from home and by solving problems.
And I was late to the party and arrived skeptical.
With crypto I arrived early (2009) and in hindsight realized that the only real value I can see from most of its existence is transfer of money to and from hard-to-reach-financially places and as an object of financial speculation.
Re: Having a 20GB file that lets you ask an offline computer any question is amazing
#25My goal: download the entire Wikipedia and store it in a vector db. Then have a local LLM answer questions based on Wikipedia. Best of both worlds!
Given the biased nature of many if not most Wikipedia articles it would be certain that such a system would bend the truth to an unacceptable degree. Train the thing on more than just Wikipedia, e.g add Encyclopedia Britannica to the training set. This is the one area where "diversity" really matters: diversity of opinion. Using a single biased source as your oracle will turn you into a pawn for those who control the…
what?
since when wikipedia is biased? and about what? (and why EB isn't?)
Re: Having a 20GB file that lets you ask an offline computer any question is amazing
#26You can download a local version of Wikipedia and yet barely anyone does this. (And I know wikipedia has a ton of problems)
Wikipedia is static, a local AI can be customized and adapted to your needs.
For example I would love an AI that could filter, summarize and prioritize news that I'm interested in.
Re: Having a 20GB file that lets you ask an offline computer any question is amazing
#27Learning is going to be revolutionized.
Re: Having a 20GB file that lets you ask an offline computer any question is amazing
#28We can do that, but then we will need to close all applications to run it because it will need 20GB RAM or VRAM to run. It is still more practical to visit stackoverflow when I have a question. (still impressive how much this improves every week. Maybe in 2-3 months we have something more useful)
I just upgraded my 2020 era thinkpad (t470, which I bought this time last year for less than £300) to 32g memory for £70.
20g of ram doesn’t feel like a lot today.
Re: Having a 20GB file that lets you ask an offline computer any question is amazing
#29I currently have a copy of offline Wikipedia (and Stack Overflow, WikiVoyage, etc.) via Kiwix on my laptop and also LLaMA-style models at the same time (since I like things being local), and with respect to asking an LLM questions, it would be great if it could leverage the offline Wikipedias to ensure it stays grounded and not make up facts.
This is why I also like to ask phind.com and chat-4. Helps keep things grounded.
Re: Having a 20GB file that lets you ask an offline computer any question is amazing
#30One of the comments on reddit
> i have no clue why everyone isn't talking about this all the time.
Talking maybe, not doing it because it's still too out of reach for most of us? I've got a laptop with 32 GB RAM and a 2GB NVIDIA card. At which speed can I run that model, if I can run it at all?