Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What's the best self hosted/local alternative to GPT-4?

news.ycombinator.com

Post reply on HN
Print view View original

191–200 of 206 posts

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#191

Post by ano88888 »

ngngngng wrote: post #125
icapybara wrote: post #120

Earlier quoted context omitted.

The LLM _is_ the product. Everything else in the stack is window dressing. I don’t think OpenAI’s contribution should be so understated- they built a technology that was considered science fiction just a few years ago. They deserve all the credit for the “AI”.

Good luck building a moat when 95% of your app is just calling the API everyone else has access to.

by this logic, you can just copy facebook, twitter and pretty all software as they are just calling some high level api (http protocol , ios/android api..) etc .

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#192

Post by raxxorraxor »

inductive_magic wrote: post #108
icapybara wrote: post #96

Earlier quoted context omitted.

This point is understated. So many people are going around like "I'm building an AI app!" when the reality is, OpenAI built an AI app, you're just designing a front end for it.

OpenAI builds an LLM and an api-interface to that model. The design of abstractions, prompt engineering, custom fine-tunes and software engineering required to ship a valuable application on top of that interface counts as "building an app" in my book.

Of course it is an app, but ultimately it is just a frontend.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#194

Post by puntofisso »

amluto wrote: post #58
_laiq wrote: post #32

Earlier quoted context omitted.

Incredible how people are underestimating ChatGPT or overestimating open-source models. A basic question: How can i join with SQL a column to a string separated with comma GPT-4: PostgreSQL: SELECT STRING_AGG(columnName, ', ') FROM tableName; Guanaco: Here is an example of how you could use the CONCAT function in MySQL to concatenate a string to a column value in a single-line query: SELECT CONCAT('The total price fo…

> How can i join with SQL a column to a string separated with comma I think I might give this one to Guanaco. I’m moderately familiar with SQL, and I can’t really understand your question. You’re faking the reader out by using “join” to mean something which is not JOIN. So do you mean to concatenate a string in a column (coming from a single row) to a “string separated with a comma”? If so (and assuming “separated” m…

I wish there was a model out there that asked me "what do you mean by 'join'" but I guess we're somewhat far from there :)

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#195

Post by poulpy123 »

ngngngng wrote: post #125
icapybara wrote: post #120

Earlier quoted context omitted.

The LLM _is_ the product. Everything else in the stack is window dressing. I don’t think OpenAI’s contribution should be so understated- they built a technology that was considered science fiction just a few years ago. They deserve all the credit for the “AI”.

Good luck building a moat when 95% of your app is just calling the API everyone else has access to.

what does this moat even mean in this context ? I see it throw around but cannot infer the meaning

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#196

Post by poulpy123 »

ano88888 wrote: post #191
ngngngng wrote: post #125

Earlier quoted context omitted.

Good luck building a moat when 95% of your app is just calling the API everyone else has access to.

by this logic, you can just copy facebook, twitter and pretty all software as they are just calling some high level api (http protocol , ios/android api..) etc .

there are plenty of FB, twitter and reddit alternative, they just don't have the data of the original. but openAI provide the API and the data

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#197

Post by poulpy123 »

pmoriarty wrote: post #165
simmanian wrote: post #163

Earlier quoted context omitted.

What tasks do you use Claude for?

I tend to use Claude primary for creative writing (at which it's better than GPT4 and even Claude+, in my experience), and for explanations, which are more thorough than GPT4 or ChatGPT (GPT 3.5) without any special prompting. Claude should be more well known, imo. ChatGPT/GPT4 gets all the hype, but Claude is really good too... sometimes even better.

it would be more known if they would provide an interface as easy to use as chatgpt.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#198

Post by berniedurfee »

puntofisso wrote: post #194
amluto wrote: post #58

Earlier quoted context omitted.

> How can i join with SQL a column to a string separated with comma I think I might give this one to Guanaco. I’m moderately familiar with SQL, and I can’t really understand your question. You’re faking the reader out by using “join” to mean something which is not JOIN. So do you mean to concatenate a string in a column (coming from a single row) to a “string separated with a comma”? If so (and assuming “separated” m…

I wish there was a model out there that asked me "what do you mean by 'join'" but I guess we're somewhat far from there :)

I think you’ve just quantified the difference between a text generator (LLM) and AGI.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#199

Post by andris9 »

CSSer wrote: post #93

There is a model that was just released called falcon-40B that is available for commercial user. It outperforms every other open LLM model available today. Buyer beware, however, because the license is custom[1] and has restrictions for "attributable revenues" over $1M/year. I'll leave that for you to interpret as you will. [0]: https://huggingface.co/tiiuae/falcon-40b-instruct [1]: https://huggingface.co/tiiuae/falc…

They changed the license for Falcon 40B today (a day later) to Apache 2.0 https://www.tii.ae/news/uaes-falcon-40b-now-royalty-free

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#200

Post by netsec_burn »

forgingahead wrote: post #140
netsec_burn wrote: post #118

Earlier quoted context omitted.

I'm running it on Serge with 51 GB of RAM. I don't think it requires any GPU memory (llama.cpp runs on the CPU), I have a RTX 2060 in the system I'm running it on. Correct me if I'm wrong.

How many tokens per second are you getting with your setup?

About .5 tokens per second if I had to guess.
Post reply on HN