If you want/need to go cpu only then llama.cpp, and the assorted front ends people are building for it, is looking like a good project: https://github.com/ggerganov/llama.cpp
I'll append this reference too, as it's related and interesting:
Ask HN: What's the best self hosted/local alternative to GPT-4?
11–20 of 206 posts
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#12> Preferably self-hosted (I'm okay with paying for it)
The big models, if even available, need >100GB of graphics memory to run and would likely take minutes to warm up.
The pricing available via OpenAI/GCP/etc is only effective when you can multi-tenant many users. The cost to run one of these systems for private use would be ~$250k per year.
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#13[flagged]
I'm all for promoting own solutions if it fits the context but OP is asking for self-hosted alternatives and this is not. This fits the definition of spam.
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#14[dead]
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#15You could hire a human to manually respond to the queries
[flagged]
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#16If you want/need to go cpu only then llama.cpp, and the assorted front ends people are building for it, is looking like a good project: https://github.com/ggerganov/llama.cpp
your statement sounds like it doesn't support GPU, llama supports GPU as well. However llama is just code to run models. OP needs to find a model that works for them, there are lots of models out there.
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#17[flagged]
I'm all for promoting own solutions if it fits the context but OP is asking for self-hosted alternatives and this is not. This fits the definition of spam.
Tbf, the OP explicitly said "Preferably self-hosted" which is why I wanted to offer this along with the explicit disclaimer that I am the founder
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#18You could hire a human to manually respond to the queries
Confidently stated responses that are often incorrect or miss the point? Just post it on /r/AskReddit and wait a little bit.
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#19https://gpt4all.io/ works fairly well on my 16 GB M1 Pro MacBook. It's certainly not on a level with ChatGPT, but what is?
It's a simple app download and allows you to select from multiple available models. No hacking required.
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#20Earlier quoted context omitted.
I'm all for promoting own solutions if it fits the context but OP is asking for self-hosted alternatives and this is not. This fits the definition of spam.
Tbf, the OP explicitly said "Preferably self-hosted" which is why I wanted to offer this along with the explicit disclaimer that I am the founder
Just stop, you are spamming on an unrelated post and now arguing about it. I’m sure people will want to sign up for your service now.