31–40 of 84 posts
I'm surprised we haven't seen dedicated boxes to self host your uncensored & private LLM yet. A bit like you can self host your apps at home on a Umbrel box. I wonder if the NVIDIA Jetson serie would be the hardware that makes the most sense?
See tinybox https://tinygrad.org/
Earlier quoted context omitted.
> knows almost everything It really doesn't. It doesn't even know what it knows and what it doesn't know. Without ways to check up on whether what it told you is true or not you may well end up in more trouble than where you were before.
How about a local wikipedia dump, with precalculated embeddings? Then you can perform a similarity search first and feed the results to the LLM. It’s less likely to hallucinate this way.
Nice achievement. How many users would realistically be able to use it at the same time when running on such a device? I am interested in its scalability.
Earlier quoted context omitted.
Yes, especially having fact checked output of LLMs would be a nice step in the right direction. Throwing out the hallucinated bits and keeping the good stuff would make LLMs a lot more applicable.
My hypothesis is that including information in the LLM’s prompt to support its answer changes the task roughly from text generation, very hallucination prone, to text summarization, or reformulation with some reasoning, and this is less likely to hallucinate. That was my personal experience in general with ChatGPT as well as LLaMa1/2.
I'm surprised we haven't seen dedicated boxes to self host your uncensored & private LLM yet. A bit like you can self host your apps at home on a Umbrel box. I wonder if the NVIDIA Jetson serie would be the hardware that makes the most sense?
I'm surprised we haven't seen dedicated boxes to self host your uncensored & private LLM yet. A bit like you can self host your apps at home on a Umbrel box. I wonder if the NVIDIA Jetson serie would be the hardware that makes the most sense?