Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What's the best self hosted/local alternative to GPT-4?

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 206 posts

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#141

Post by whimsicalism »

inductive_magic wrote: post #136
icapybara wrote: post #120

Earlier quoted context omitted.

The LLM _is_ the product. Everything else in the stack is window dressing. I don’t think OpenAI’s contribution should be so understated- they built a technology that was considered science fiction just a few years ago. They deserve all the credit for the “AI”.

I wasn't trying to understate OpenAI's contribution, far from it. I can hardly express my appreciation for their work. That doesn't mean that everything else in the stack is window dressing though - custom, domain specific wrangling with the different api endpoints, finding a satisfying prompt, temperature param etc. for specific tasks - the entire process of designing systems around an LLM-api has many intricacies t…

I am not convinced it won’t all be obviated with improvements in the foundation model. Seems to be carving out a very very temporary space for yourself.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#142

Post by ngngngng »

forgingahead wrote: post #137
ngngngng wrote: post #125

Earlier quoted context omitted.

Good luck building a moat when 95% of your app is just calling the API everyone else has access to.

Paul English (founder of Kayak, exited at $2 billion) said this in response to the exact comment you are making: "Kayak was just a thin layer above ITA, and ITA sold for $700m and Kayak sold for $2b. So don’t dismiss thin layers. The ultimate power is the app and UX."[0] I would also add "distribution" to the "app and UX" part, but you can certainly build a valuable business upon an API "everyone else has access to"…

I can appreciate this, but there's a reason I said 95%. The stuff I see being built so far is really an extremely thin veneer on top of the API, but that could change with time.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#143

Post by discordance »

ivanhoe wrote: post #113
_laiq wrote: post #32

Earlier quoted context omitted.

Incredible how people are underestimating ChatGPT or overestimating open-source models. A basic question: How can i join with SQL a column to a string separated with comma GPT-4: PostgreSQL: SELECT STRING_AGG(columnName, ', ') FROM tableName; Guanaco: Here is an example of how you could use the CONCAT function in MySQL to concatenate a string to a column value in a single-line query: SELECT CONCAT('The total price fo…

> GPT-4: PostgreSQL: SELECT STRING_AGG(columnName, ', ') FROM tableName; Well, you asked for sql and ansi sql function is listagg(), so technically this is not the right answer - and definitely a completely useless one for mysql or oracle users.

"Thanks ChatGPT. Could you please write that for MySQL and additionally for Oracle?"

MySQL: SELECT GROUP_CONCAT(columnName SEPARATOR ', ') FROM tableName;

Oracle: SELECT LISTAGG(columnName, ', ') WITHIN GROUP (ORDER BY columnName) FROM tableName;

I mean, it's not a big deal.. you just give have to give it a little nudge.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#144

Post by sharkjacobs »

icapybara wrote: post #120
inductive_magic wrote: post #108

Earlier quoted context omitted.

OpenAI builds an LLM and an api-interface to that model. The design of abstractions, prompt engineering, custom fine-tunes and software engineering required to ship a valuable application on top of that interface counts as "building an app" in my book.

The LLM _is_ the product. Everything else in the stack is window dressing. I don’t think OpenAI’s contribution should be so understated- they built a technology that was considered science fiction just a few years ago. They deserve all the credit for the “AI”.

> The engine _is_ the product. Everything else in the car is window dressing.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#145

Post by whimsicalism »

forgingahead wrote: post #137
ngngngng wrote: post #125

Earlier quoted context omitted.

Good luck building a moat when 95% of your app is just calling the API everyone else has access to.

Paul English (founder of Kayak, exited at $2 billion) said this in response to the exact comment you are making: "Kayak was just a thin layer above ITA, and ITA sold for $700m and Kayak sold for $2b. So don’t dismiss thin layers. The ultimate power is the app and UX."[0] I would also add "distribution" to the "app and UX" part, but you can certainly build a valuable business upon an API "everyone else has access to"…

Kayak would never have succeeded today. Competition is far too hungry

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#146

Post by whimsicalism »

PeterisP wrote: post #117
egonschiele wrote: post #81

Earlier quoted context omitted.

I believe there are benchmarks, but I can informally second that opinion. I'm building a writing app (chiseleditor.com) and there is nothing as good as the ChatGPT models right now.

I would distrust the currently available benchmarks, as recent research (gah, can't remember the paper title) indicates that for many benchmarks at least some of the data splits have leaked into model training data; and there's some experience with the open source models which match an OpenAI model on the benchmark scores but subjectively feel much worse than that model on random questions.

I’m telling you from looking at this closely that there is substantial evidence solely from new, never seen before prompts that GPT4 is by far the best, ChatGPT/Claude is second, with other anthropic, Vicuña, etc bringing up the rear

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#147

Post by golergka »

icapybara wrote: post #120
inductive_magic wrote: post #108

Earlier quoted context omitted.

OpenAI builds an LLM and an api-interface to that model. The design of abstractions, prompt engineering, custom fine-tunes and software engineering required to ship a valuable application on top of that interface counts as "building an app" in my book.

The LLM _is_ the product. Everything else in the stack is window dressing. I don’t think OpenAI’s contribution should be so understated- they built a technology that was considered science fiction just a few years ago. They deserve all the credit for the “AI”.

I think that people who believe this don't realise how complex and nuanced code surrounding LLMs have gotten in the last half a year.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#148

Post by whimsicalism »

edgyquant wrote: post #135
simonw wrote: post #69

Earlier quoted context omitted.

I just got "The capital of France is Paris." from vicuna-v1-7B running entirely on my iPhone (using the MLC Chat app).

>They often get Paris right at least, but most other details are wrong.

What other detail in that statement is wrong?

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#149

Post by colesantiago »

jfoster wrote: post #128
colesantiago wrote: post #77

The best self hosted/local alternative to GPT-4 is a (self hosted) GPT-X variant by OpenAI. No kidding, and I am calling it on the record right here. OpenAI will release an 'open source' model to try and recoup their moat in the self hosted / local space. https://www.theinformation.com/briefings/openai-readies-new-...

Does it benefit OpenAI if people are using an "open source" OpenAI model versus any other "open source" model?

Great question.

Yes it does benefit OpenAI because Sam is pushing for and betting on regulatory capture. Meaning AI models that are compliant with AI safety principles would be allowed and considered safe by law should regulations be put in place.

Current open source models wouldn't be compliant and would require work to make them compliant, thus creating a moat for OpenAI in the self hosted space.

Because there is nothing better than a free model that you can self host that OpenAI has made that is also regulated.

Enterprises would love this, since OpenAI has the mindshare already, they can self host a licensed model in their orgs or air gapped environment that they know is regulated.

OpenAI may release different model sizes of this GPT-X variant like they did quietly for Whisper, Bigger sizes may require an enterprise license.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#150

Post by belter »

wokwokwok wrote: post #30

There is literally no alternative. You’re stuck with openai, and you’re stuck with whatever rules, limitations or changes they give you. There are other models, but specifically if you’re actively using gpt-4 and find gpt-3.5 to be below the quality you require… Too bad. You’re out of luck. Wait for better open source models or wait patiently for someone to release a meaningful competitor, or wait for openai to relea…

There is no AI, there are LLM models, that people hardly understand. Modern solutions are to AI what Alchemy was to Chemistry. Sometimes they get some results but nobody really sure why. The little that OpenAI mentioned about how they achieved GPT-4 performance, says it was by trying a bunch of things and dismissing others. They probably also themselves not sure how they managed to pull it of, meaning that, if for example an increase in parameter size or other settings, kick in negative effects that actually reduce the model performance, they will also be at a loss where to go next.
Post reply on HN