Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What's the best self hosted/local alternative to GPT-4?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 206 posts

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#71

Post by croes »

_laiq wrote: post #32
jonathan-adly wrote: post #25

I don't know the licensing and all that jazz (even if you self-host for your personal use it shouldn't matter). But, this paper[0] released a week ago claims " 99.3% of the performance level of ChatGPT while only requiring 24 hours of finetuning on a single GPU" (QLORA). A quick test of the huggingface demo gives reasonable results[1]. The actual model behind the space is here[2], and should be self-hostable with rea…

Incredible how people are underestimating ChatGPT or overestimating open-source models. A basic question: How can i join with SQL a column to a string separated with comma GPT-4: PostgreSQL: SELECT STRING_AGG(columnName, ', ') FROM tableName; Guanaco: Here is an example of how you could use the CONCAT function in MySQL to concatenate a string to a column value in a single-line query: SELECT CONCAT('The total price fo…

What's the result for: How can I aggregate with SQL a column to a string separated with comma ?

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#72

Post by VadimPR »

MacsHeadroom wrote: post #54
AndroTux wrote: post #19

https://gpt4all.io/ works fairly well on my 16 GB M1 Pro MacBook. It's certainly not on a level with ChatGPT, but what is? It's a simple app download and allows you to select from multiple available models. No hacking required.

> It's certainly not on a level with ChatGPT, but what is? Guanaco-65B is per https://arxiv.org/abs/2305.14314 CPU Version: https://huggingface.co/TheBloke/guanaco-65B-GGML GPU Version: https://huggingface.co/TheBloke/guanaco-65B-HF 4bit GPU Version: https://huggingface.co/TheBloke/guanaco-65B-GPTQ

How much ram/vram required?

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#73

Post by sgd99 »

Not self-hosted/local but Claude by Anthropic from what I've heard is really good but the API is not publicly available. It's apparently accessible via Poe (https://poe.com)

As for open models, HuggingFace has a nice leaderboard to see which ones are decent: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#75

Post by davepeck »

There are no viable self-hostable alternatives to GPT-4 or even to GPT3.5.

The “best” self-hostable model is a moving target. As of this writing it’s probably one of Vicuña 13B, Wizard 30B, or maybe Guanaco 65B. I’d like to say that Guanaco is wildly better than Vicuña, what with its 5x larger size. But… that seems very task dependent.

As anecdata: my experience is that none of these is as good as even GPT3.5 for summarization, extraction, sentiment analysis, or assistance with writing code. Figuring out how to run them is painful. The speed at which their unquantized variants run on any hardware I have access to is painful. Sorting through licensing is… also painful.

And again: they’re nowhere close to GPT-4.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#76

Post by nauful »

_laiq wrote: post #32
jonathan-adly wrote: post #25

I don't know the licensing and all that jazz (even if you self-host for your personal use it shouldn't matter). But, this paper[0] released a week ago claims " 99.3% of the performance level of ChatGPT while only requiring 24 hours of finetuning on a single GPU" (QLORA). A quick test of the huggingface demo gives reasonable results[1]. The actual model behind the space is here[2], and should be self-hostable with rea…

Incredible how people are underestimating ChatGPT or overestimating open-source models. A basic question: How can i join with SQL a column to a string separated with comma GPT-4: PostgreSQL: SELECT STRING_AGG(columnName, ', ') FROM tableName; Guanaco: Here is an example of how you could use the CONCAT function in MySQL to concatenate a string to a column value in a single-line query: SELECT CONCAT('The total price fo…

I have a hard time understanding the question as you phrased it. Tested with guanaco-33B and a cleaner question, looks like a much better response:

> How can I join a column with SQL in Postgres to a string separated with a comma?

You can use the `string_agg` function in PostgreSQL to concatenate columns into a string. For example, to concatenate all values from a column named "column1" into a single string separated by commas, you can run the following query:

```sql

SELECT string_agg(column1, ',') FROM table;

```

The `string_agg` function takes two arguments: the first argument is the expression to concatenate (in this case column1), and the second argument is the separator between each value (in this case a comma). The query above will return a single string containing all values from column1 separated by commas.

I generally find WizardLM 1.0, Guanaco and Manticore to be good enough for most GPT cases.

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#77

Post by colesantiago »

The best self hosted/local alternative to GPT-4 is a (self hosted) GPT-X variant by OpenAI.

No kidding, and I am calling it on the record right here.

OpenAI will release an 'open source' model to try and recoup their moat in the self hosted / local space.

https://www.theinformation.com/briefings/openai-readies-new-...

Re: Ask HN: What's the best self hosted/local alternative to GPT-4?

#79

Post by zorrobyte »

What's the best self hosted for ingesting a local codebase and wiki to ask questions of it? Some of the projects linked here have ingest scripts for doc, pdf files; but it'd be cool to ingest a whole git repo and wiki, have a little chat interface to ask questions about the code.
Post reply on HN