Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What's the best self hosted/local alternative to GPT-4?

news.ycombinator.com

Post reply on HN
Print view View original

91–100 of 206 posts

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#91

Post by findalex »

rhacker wrote: post #56
wokwokwok wrote: post #30

There is literally no alternative. You’re stuck with openai, and you’re stuck with whatever rules, limitations or changes they give you. There are other models, but specifically if you’re actively using gpt-4 and find gpt-3.5 to be below the quality you require… Too bad. You’re out of luck. Wait for better open source models or wait patiently for someone to release a meaningful competitor, or wait for openai to relea…

Guess there really is a moat.

Compute moat. aka cash moat. It's a deep one, too.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#92

Post by RecycledEle »

I often worry about aa "The Machine Stops" scenario.

GPT AI actually gives me hope. What if we can store and run an AI in a phone-sized-device that is superior to a similarly sized library of books? Can we have a rugged, solar-powered device that could survive the fall of Civilization and help us rebuild?

It would certainly have military applications in a warfare. Imagine being the 21ct century equivalent of a 1940's US Marine on Guadal Canal who need to know some survival skills. ChatGPT-on-a-phone would be handy if you could keep the battery charged.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#93

Post by CSSer »

There is a model that was just released called falcon-40B that is available for commercial user. It outperforms every other open LLM model available today. Buyer beware, however, because the license is custom[1] and has restrictions for "attributable revenues" over $1M/year. I'll leave that for you to interpret as you will.

[0]: https://huggingface.co/tiiuae/falcon-40b-instruct [1]: https://huggingface.co/tiiuae/falcon-40b-instruct/blob/main/...

EDIT: I just realized you seem to be asking for a fully realized, turn-key commercial solution. Yeah, refer to others who say there's no alternative. It's true. Something like this gives you a lot more power and flexibility, but at the cost of a lot more work building the solution as you try to apply it.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#94

Post by RC_ITR »

amluto wrote: post #58
_laiq wrote: post #32

Earlier quoted context omitted.

Incredible how people are underestimating ChatGPT or overestimating open-source models. A basic question: How can i join with SQL a column to a string separated with comma GPT-4: PostgreSQL: SELECT STRING_AGG(columnName, ', ') FROM tableName; Guanaco: Here is an example of how you could use the CONCAT function in MySQL to concatenate a string to a column value in a single-line query: SELECT CONCAT('The total price fo…

> How can i join with SQL a column to a string separated with comma I think I might give this one to Guanaco. I’m moderately familiar with SQL, and I can’t really understand your question. You’re faking the reader out by using “join” to mean something which is not JOIN. So do you mean to concatenate a string in a column (coming from a single row) to a “string separated with a comma”? If so (and assuming “separated” m…

> You’re faking the reader out by using “join” to mean something which is not JOIN.

Yeah and not having to do precise prompt engineering is a major advantage of GPT-4 vs other models.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#95

Post by gpderetta »

RC_ITR wrote: post #94
amluto wrote: post #58

Earlier quoted context omitted.

> How can i join with SQL a column to a string separated with comma I think I might give this one to Guanaco. I’m moderately familiar with SQL, and I can’t really understand your question. You’re faking the reader out by using “join” to mean something which is not JOIN. So do you mean to concatenate a string in a column (coming from a single row) to a “string separated with a comma”? If so (and assuming “separated” m…

> You’re faking the reader out by using “join” to mean something which is not JOIN. Yeah and not having to do precise prompt engineering is a major advantage of GPT-4 vs other models.

To be fair, I had to read the postgresql docs and interpret the answer to figure out what the _laiq was actually asking.

edit: I guess GPT-4 has better reading comprension than me!

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#96

Post by icapybara »

wokwokwok wrote: post #30

There is literally no alternative. You’re stuck with openai, and you’re stuck with whatever rules, limitations or changes they give you. There are other models, but specifically if you’re actively using gpt-4 and find gpt-3.5 to be below the quality you require… Too bad. You’re out of luck. Wait for better open source models or wait patiently for someone to release a meaningful competitor, or wait for openai to relea…

This point is understated. So many people are going around like "I'm building an AI app!" when the reality is, OpenAI built an AI app, you're just designing a front end for it.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#97

Post by simonh »

rhacker wrote: post #56
wokwokwok wrote: post #30

There is literally no alternative. You’re stuck with openai, and you’re stuck with whatever rules, limitations or changes they give you. There are other models, but specifically if you’re actively using gpt-4 and find gpt-3.5 to be below the quality you require… Too bad. You’re out of luck. Wait for better open source models or wait patiently for someone to release a meaningful competitor, or wait for openai to relea…

Guess there really is a moat.

Just to nitpick that a bit, they have a lead. To me, a moat is some other axis of advantage that makes it hard for a rival to compete directly.

So for example Tesla has/had a lead in EV tech, but their supercharger network is a moat because it’s very difficult for a rival to compete with, even if they have equivalent EV tech.

I know there isn’t established terminology for this. It is a nitpick, but I think a lead is already a term we have, and moat has connotations of being in some way ‘unfair’. To me, just creating a better product isn’t unfair in any meaningful way, while moats in some circumstances can represent anti trust issues.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#98

Post by orost »

wing-_-nuts wrote: post #85
MacsHeadroom wrote: post #65

Earlier quoted context omitted.

You link to Guanaco-33B, but Guanaco-65B is much more capable. CPU Version: https://huggingface.co/TheBloke/guanaco-65B-GGML GPU Version: https://huggingface.co/TheBloke/guanaco-65B-HF 4bit GPU Version: https://huggingface.co/TheBloke/guanaco-65B-GPTQ

It irritates me to no end that people don't list the system requirements of various models. How much ram and vram does one need to run 4,13,33,65B models at a reasonable speed? edit: instead I'll ask this, what's the best model to run on a system with a 24gb 4090 and 64gb of ram?

You can just barely fit a 33B GPTQ model in 24GB VRAM. It will be in 4-bit mode, and without maximum context size, but it will be quite fast. Or you can run from RAM+VRAM in GGML format with llama.cpp (or a derivative), which will easily fit 65B models even at 5 or 8 bits, but at much lower speed.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#100

Post by amluto »

RC_ITR wrote: post #94
amluto wrote: post #58

Earlier quoted context omitted.

> How can i join with SQL a column to a string separated with comma I think I might give this one to Guanaco. I’m moderately familiar with SQL, and I can’t really understand your question. You’re faking the reader out by using “join” to mean something which is not JOIN. So do you mean to concatenate a string in a column (coming from a single row) to a “string separated with a comma”? If so (and assuming “separated” m…

> You’re faking the reader out by using “join” to mean something which is not JOIN. Yeah and not having to do precise prompt engineering is a major advantage of GPT-4 vs other models.

I don’t think that “prompt engineering” is quite the same thing as clearly specifying the problem. If I’m using a magic AI code-writing companion or asking an actual human to write code for me, I need to define the problem well enough if I want any expectation of getting _correct_ code.

A skilled human will distinguish themselves from a less skilled human by also thinking about the overall problem and helping make sure the specification is correct. This far, I have not seen an AI meet that bar unless the problem is something well-covered already on the web.

I have seen ChatGPT (3.5 and 4) output code that is syntactically correct, executes without errors, and does the wrong things in ways that are not entirely obvious to me until I read the docs myself.

Post reply on HN