Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The genie escapes: Stanford copies the ChatGPT AI for less than $600

newatlas.com

Post reply on HN
Print view View original

31–40 of 173 posts

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#31

Post by stavros »

ronsor wrote: post #5
xwdv wrote: post #2

Given the high prices of OpenAI offerings it seems it’s better to pirate an AI model before resorting to paying for anything.

ChatGPT API is surprisingly cheap, but GPT-4 is many times more expensive to the point where I can't see it being worth it most of the time.

I'm not convinced that the GPT-4 API actually works? It's been giving me very different answers than the chat interface. For example, the chat interface says it's GPT-4 if you ask it, but the API says it's GPT-3 (and bills as GPT-4).

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#32

Post by stavros »

LASR wrote: post #20
superkuh wrote: post #8

Hardly. I've played a lot with the 7,13, and 30B llamas as well as the 7 and 13B alpacas fine tuned by Stanford. They do not have emergent abilities like being able to generate rhymes or, say, represent a movie plot as emoji. Even openai's old text-davinci-003 (gpt3.5, but text completion, not the chat ones) far outperforms them. That said, I have hopes for a 65B 3-bit quantized alpaca-fine tuned. We'll see when some…

In my personal testing, I throw some sophisticated use cases at LLMs - particularly chain of thought reasoning. None of the models out there are able to do this this well, except for the OG GPT-3 Davinci-003. Even the newer turbo models are not as good. I am playing around with GPT-4 this week though. Let’s see how that goes.

The newer turbo models are the ChatGPT models, and are worse than text-davinci-003, in my experience. The gpt-4 model is also not as good as the GPT-4 chat version, which is very odd.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#33

Post by crooked-v »

jeron wrote: post #13
twblalock wrote: post #6

This is why it's not possible to slow down or "stop" AI: once the problems are solved the solutions turn out to be trivial to replicate. All it takes is compute.

you say all it takes compute like that is trivial - chatGPT would have a hard time without Microsoft's support via Azure

While that's true, it's basically inevitable now that at some point personal hardware will be powerful enough for enthusiasts to run home bots comparable to GPT-3, and even that by itself would drastically change a lot things.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#34

Post by sp332 »

Firmwarrior wrote: post #21
xwdv wrote: post #2

Given the high prices of OpenAI offerings it seems it’s better to pirate an AI model before resorting to paying for anything.

Isn't openAI only charging something like a 20th of a penny per interaction right now? Definitely not the kind of thing you want to incorporate into a widespread free app just yet, but it seems pretty affordable for a lot of use cases

That's the price per "token". A token is a word or part of a word - rule of thumb is four tokens per three words.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#35

Post by dragonwriter »

Firmwarrior wrote: post #21
xwdv wrote: post #2

Given the high prices of OpenAI offerings it seems it’s better to pirate an AI model before resorting to paying for anything.

Isn't openAI only charging something like a 20th of a penny per interaction right now? Definitely not the kind of thing you want to incorporate into a widespread free app just yet, but it seems pretty affordable for a lot of use cases

> Isn't openAI only charging something like a 20th of a penny per interaction right now?

They don't charge per interaction, but per token. The chat models range from a fifth of a cent per 1000 tokens to 12 cents per thousand tokens (depending on whether it's gpt-3.5, or the 8k limit gpt-4, or the 32k limit gpt-4, and, for gpt-4 models, also prompt v. response tokens.)

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#36

Post by [deleted] »

thomasahle wrote: post #29
superkuh wrote: post #8

Hardly. I've played a lot with the 7,13, and 30B llamas as well as the 7 and 13B alpacas fine tuned by Stanford. They do not have emergent abilities like being able to generate rhymes or, say, represent a movie plot as emoji. Even openai's old text-davinci-003 (gpt3.5, but text completion, not the chat ones) far outperforms them. That said, I have hopes for a 65B 3-bit quantized alpaca-fine tuned. We'll see when some…

3 bits? Is that for all weights in the network?

[deleted]

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#37

Post by superkuh »

thomasahle wrote: post #29
superkuh wrote: post #8

Hardly. I've played a lot with the 7,13, and 30B llamas as well as the 7 and 13B alpacas fine tuned by Stanford. They do not have emergent abilities like being able to generate rhymes or, say, represent a movie plot as emoji. Even openai's old text-davinci-003 (gpt3.5, but text completion, not the chat ones) far outperforms them. That said, I have hopes for a 65B 3-bit quantized alpaca-fine tuned. We'll see when some…

3 bits? Is that for all weights in the network?

As far as I know, yes. https://arxiv.org/abs/2210.17323

"Specifically, GPTQ can quantize GPT models with 175 billion parameters in approximately four GPU hours, reducing the bitwidth down to 3 or 4 bits per weight, with negligible accuracy degradation relative to the uncompressed baseline."

This would be 175 billion 3 bit weights instead of 175 billion 16 (or 32!) bit weights. It massively reduces the size of the model. It makes loading it in ram on consumer computers feasible. The number of parameters stays the same.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#39

Post by siva7 »

stavros wrote: post #31
ronsor wrote: post #5

Earlier quoted context omitted.

ChatGPT API is surprisingly cheap, but GPT-4 is many times more expensive to the point where I can't see it being worth it most of the time.

I'm not convinced that the GPT-4 API actually works? It's been giving me very different answers than the chat interface. For example, the chat interface says it's GPT-4 if you ask it, but the API says it's GPT-3 (and bills as GPT-4).

If you're satisfied with the results who cares what it thinks it is

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#40

Post by layer8 »

crooked-v wrote: post #33
jeron wrote: post #13

Earlier quoted context omitted.

you say all it takes compute like that is trivial - chatGPT would have a hard time without Microsoft's support via Azure

While that's true, it's basically inevitable now that at some point personal hardware will be powerful enough for enthusiasts to run home bots comparable to GPT-3, and even that by itself would drastically change a lot things.

Running isn’t necessarily the issue. The moat is creating a high-quality model like OpenAI has, which (and here the article is mistaken) doesn’t seem to be easily reproducible.
Post reply on HN