131–140 of 140 posts
Earlier quoted context omitted.
a primer - https://github.com/nilsherzig/LLocalSearch/issues/17
guys, i didn't thought there would be this much interest in my project haha. I feel kinda bad for just posting it in this state haha. I would love to make a more detailed post on how it works in the future (keep an eye on the repo?)
Earlier quoted context omitted.
If you're open to it, it would be great if you could make a post explaining how you built this. Even if it's brief. Trying to learn more about this space and this looks pretty cool. And ofc, nice work!
a primer - https://github.com/nilsherzig/LLocalSearch/issues/17
Btw. This is the first time I hear about Perplexity, which after 10 minutes of experimentation, looks like a worse clone of Phind.
In five year's time - by 2030, I foresee that lots of inference would be happening on local machines with models being downloaded on demand. Think docker registry of AI models which is pretty much Hugging Face already there. This all would be due to optimisations within model inference code and techniques, hardware and packaging of software like the above. Don't see billion dollar valuations for lots of AI startups o…
That's why I think these private companies will have the best AIs for many decades.
Earlier quoted context omitted.
> I foresee that lots of inference would be happening on local machines with models being downloaded on demand Why? It's much more efficient to have centralized special purpose hardware to run enormous models and then ship the comparatively small result over the internet. By analogy, you don't have a search engine running on your phone right?
A more appropriate analogy would be driving your own car vs. taking the bus.
Will not happen any time soon. Consumer hardware can't even run GPT-4 locally, and won't be able for a looong time. Each GPT-4 instance runs on 8 A100. The cost of such system is ~$81K. Not even in the ballpark of what most consumers can afford.
Speaking of LLM's... here's my "dear lazyweb" to HN: What would be the best self hosted option to build sort of a textual AI assistant into your app? Preferably something that I can train myself over time with domain knowledge.
Fine tuning on your own knowledge probably isn't what you want to do, you probably want to do retrieval aided generation instead. Basically a search engine on some local documents, and you put the results of the search into your prompt. The search engine uses the same vector space as your language model as its index, so the results should be highly relevant to whatever the prompt is. I'd start with "librechat" and mi…
Impressive, I don't think I've seen a local model call upon specialised modules yet (although I can't keep up with everything going on). I too use local 7b open-hermes and it's really good.
Thanks :). It's just a lot of prompting and string parsing. There are models like "Hermes-2-Pro-Mistral" (the one from the video) which are trained to work with function signatures and outputting structured text. But at the end it's just strings in > strings out, haha. But its fun (and sometimes frustrating) to use LLMs for flow control (conditions, loops...) inside your programs.