Earlier quoted context omitted.
100% of all existing code is just calling an OS-API that everyone else has access to. That's just nonsense
Yes, but also, come on now.
[dead]
Ask HN: What's the best self hosted/local alternative to GPT-4?
161–170 of 206 posts
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#162Earlier quoted context omitted.
I wasn't trying to understate OpenAI's contribution, far from it. I can hardly express my appreciation for their work. That doesn't mean that everything else in the stack is window dressing though - custom, domain specific wrangling with the different api endpoints, finding a satisfying prompt, temperature param etc. for specific tasks - the entire process of designing systems around an LLM-api has many intricacies t…
I am not convinced it won’t all be obviated with improvements in the foundation model. Seems to be carving out a very very temporary space for yourself.
Re: smart people avoiding wrapper work - sure! Not just because the result might be short lived, but also because there is a lot more prestige in building something from scratch rather than wrapping.
However, the "very very temporary space" might as well lead to momentum and a moat in a subdomain, and anyone who met a sufficiently large number of smart people knows that lots of them are very pragmatic, don't chase prestige, and enjoy laying ground work for future iterations.
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#163There is literally no alternative. You’re stuck with openai, and you’re stuck with whatever rules, limitations or changes they give you. There are other models, but specifically if you’re actively using gpt-4 and find gpt-3.5 to be below the quality you require… Too bad. You’re out of luck. Wait for better open source models or wait patiently for someone to release a meaningful competitor, or wait for openai to relea…
Anthropic's Claude is on par with or better than GPT-4 for many tasks. It's not self-hosted, but it is competitive with OpenAI's best model.
What tasks do you use Claude for?
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#164Earlier quoted context omitted.
I believe there are benchmarks, but I can informally second that opinion. I'm building a writing app (chiseleditor.com) and there is nothing as good as the ChatGPT models right now.
Have you tried Anthropic, specifically Claude? I have no doubt GPT-4 is still king, I'm just curious how much of a lead it has.
I've played around a lot with Claude, and find it much better than GPT4 and even Claude+ at creative writing.
I also generally prefer Claude or Claude+ over GPT4 or ChatGPT (GPT 3.5) for explanations too, which tend to be more thorough without any special prompting.
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#165Earlier quoted context omitted.
Anthropic's Claude is on par with or better than GPT-4 for many tasks. It's not self-hosted, but it is competitive with OpenAI's best model.
What tasks do you use Claude for?
I tend to use Claude primary for creative writing (at which it's better than GPT4 and even Claude+, in my experience), and for explanations, which are more thorough than GPT4 or ChatGPT (GPT 3.5) without any special prompting.
Claude should be more well known, imo. ChatGPT/GPT4 gets all the hype, but Claude is really good too... sometimes even better.
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#166Earlier quoted context omitted.
100% of all existing code is just calling an OS-API that everyone else has access to. That's just nonsense
Yes, but also, come on now.
I mean, it refutes the argument. Everyone has access to the internet, to conv nets, to C/python/pandas/TCP, you name it. Yet, nobody would seriously argue that one can not build a moat with products based on those stacks for the reason that everyone else has access to the same stack. Its just not an argument, at all.
We can now write software that interprets language under the hood (to some degree). The value propositions enabled by this change in the world are so vast, and partly so complex - to make absolute statements like "yeah but you don't control the model, so anyone can copy your solution" seems out of touch to me. What subset of technology doesn't get reverse engineered? Either this applies almost nowhere (because every piece of tech that an engineer can get their hands on is effectively open), or everywhere.
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#167Earlier quoted context omitted.
What tasks do you use Claude for?
I tend to use Claude primary for creative writing (at which it's better than GPT4 and even Claude+, in my experience), and for explanations, which are more thorough than GPT4 or ChatGPT (GPT 3.5) without any special prompting. Claude should be more well known, imo. ChatGPT/GPT4 gets all the hype, but Claude is really good too... sometimes even better.
I thought Anthropic has Claude and Claude Instant. When you say Claude sometimes feels better than Claude+, do you mean that you feel Claude Instant sometimes gives better result than Claude?
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#168I don't know the licensing and all that jazz (even if you self-host for your personal use it shouldn't matter). But, this paper[0] released a week ago claims " 99.3% of the performance level of ChatGPT while only requiring 24 hours of finetuning on a single GPU" (QLORA). A quick test of the huggingface demo gives reasonable results[1]. The actual model behind the space is here[2], and should be self-hostable with rea…
Incredible how people are underestimating ChatGPT or overestimating open-source models. A basic question: How can i join with SQL a column to a string separated with comma GPT-4: PostgreSQL: SELECT STRING_AGG(columnName, ', ') FROM tableName; Guanaco: Here is an example of how you could use the CONCAT function in MySQL to concatenate a string to a column value in a single-line query: SELECT CONCAT('The total price fo…
One of the issues that i've observed with OpenAI, is that things the model got wrong a week ago it suddenly starts getting right but other examples that are not exactly the same question but very similar or another example of the same thing it gets wrong, then a few weeks passes and it gets this right.
My conclusion is that they are using the data from users to improve or some other prompt injection tricks.
So the comparision is not even fair.
Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#169Re: Ask HN: What's the best self hosted/local alternative to GPT-4?
#170I don't know the licensing and all that jazz (even if you self-host for your personal use it shouldn't matter). But, this paper[0] released a week ago claims " 99.3% of the performance level of ChatGPT while only requiring 24 hours of finetuning on a single GPU" (QLORA). A quick test of the huggingface demo gives reasonable results[1]. The actual model behind the space is here[2], and should be self-hostable with rea…
Guanaco is indeed very capable and can replace GPT 3.5 in almost all scenarios, based on my tests.
Easy way to self-host it is to use text-generation-webui[1] and 33B 4-bit quantized GGML model from TheBloke[2].