Ask HN: What's the best self hosted/local alternative to GPT-4?

Constant outages and the model seemingly getting nerfed[^1] are driving me insane. Which viable alternatives to GPT-4 exist? Preferably self-hosted (I'm okay with paying for it) and with an API that's compatible with the OpenAI API.

[^1]: https://news.ycombinator.com/item?id=36134249

How much GPU memory do you have access to? If you can run it, Guanaco-65B is probably as close as you can get in terms of something publicly available. https://github.com/artidoro/qlora. But as other comments mention, it's still noticeably worse in my experience.

OpenAssistant is pretty good. It still has some censorship but nowhere near the levels of commercial models.

It’s actually impressive how good it is considering the limited resources they have.

The answer to this question changes every week.

For compatibility with the OpenAI API one project to consider is https://github.com/go-skynet/LocalAI

None of the open models are close to GPT-4 yet, but some of the LLaMA derivatives feel similar to GPT3.5.

Licenses are a big question though: if you want something you can use for commercial purposes your options are much more limited.

This is a good candidate: https://github.com/imartinez/privateGPT