Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The genie escapes: Stanford copies the ChatGPT AI for less than $600

newatlas.com

Post reply on HN
Print view View original

101–110 of 173 posts

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#101

Post by trevyn »

stavros wrote: post #88
trevyn wrote: post #84

Earlier quoted context omitted.

Has the ChatGPT-4 “system” prompt been revealed?

Hmm, you think that's what makes a difference? It might be, though I've gotten different (what I find worse) results on other questions as well.

This is a great (pre-GPT-4) post that goes into the system prompt and its effects: https://minimaxir.com/2023/03/new-chatgpt-overlord/

Are you using any system prompt in your API calls? Note that it does have an explicit domain separation from the user and assistant roles.

I suspect it plays a greater role in GPT-4 as well, i.e. was trained on a wider variety of system prompts to guide the output of the model.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#102

Post by tantalor »

EGreg wrote: post #75

I warned about this for years. Finally an article gets it right. Everyone will soon have the equivalent of online nuclear weapons: bot swarms that infiltrate every forum, including this one.

Spam has existed on the internet for a long time.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#103

Post by sebzim4500 »

rnosov wrote: post #49
superkuh wrote: post #37

Earlier quoted context omitted.

As far as I know, yes. https://arxiv.org/abs/2210.17323 "Specifically, GPTQ can quantize GPT models with 175 billion parameters in approximately four GPU hours, reducing the bitwidth down to 3 or 4 bits per weight, with negligible accuracy degradation relative to the uncompressed baseline." This would be 175 billion 3 bit weights instead of 175 billion 16 (or 32!) bit weights. It massively reduces the size of the mod…

> https://arxiv.org/abs/2210.17323 I've read the paper and to be honest I'm not sure what to make of it. Their headline benchmark is perplexity on WikiText2 which would not be particularly relevant to most users. If you look at the tables in the appendix A.4 with some more relevant benchmarks you'll sometimes find that straight RTN 4 bit quantisation beats both GPTQ and even full 16 bit original! No explanation of it…

Some of those benchmarks have a pretty small sample size IIRC, might just be coincidence that the noise introduced by RTN just happens to slightly improve them.

GPTQ beats RTN on almost every benchmark at almost every size, though.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#104

Post by simonw »

f_devd wrote: post #76
doctoboggan wrote: post #50

I've used both the 7B and 13B instruction tuned llama weights (quantized using the llama.cpp scripts). Either I am doing something wrong, or these two models are no-where near the level of ChatGPT. Many times they return something totally irrelevant to my question, stop responding, use a different language, or otherwise return the wrong answer. ChatGPT does none of this. (other than the wrong answer due to hallucinat…

LLama hasn't been fine-tuned with RLHF, so it requires additional prompting, check out the open-assistant[0] project for an open-source ChatGPT equivalent (WIP). [0]: https://github.com/LAION-AI/Open-Assistant

This is why Alpaca is a big deal: it shows what LLaMA can do after it's been fine-tuned to follow instructions like ChatGPT has.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#105

Post by genericacct »

superkuh wrote: post #8

Hardly. I've played a lot with the 7,13, and 30B llamas as well as the 7 and 13B alpacas fine tuned by Stanford. They do not have emergent abilities like being able to generate rhymes or, say, represent a movie plot as emoji. Even openai's old text-davinci-003 (gpt3.5, but text completion, not the chat ones) far outperforms them. That said, I have hopes for a 65B 3-bit quantized alpaca-fine tuned. We'll see when some…

fwiw 7B is totaly useless for the subset of non english languages i've used, 13B a bit less so, but nowhere near as good as gpt.

GPT's performance in non-trivial translation tasks is unbelievable. all those articles mentioning jobs that are going to be replaced fail to mention translators are probably going to be the first.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#106

Post by onlyrealcuzzo »

superkuh wrote: post #8

Hardly. I've played a lot with the 7,13, and 30B llamas as well as the 7 and 13B alpacas fine tuned by Stanford. They do not have emergent abilities like being able to generate rhymes or, say, represent a movie plot as emoji. Even openai's old text-davinci-003 (gpt3.5, but text completion, not the chat ones) far outperforms them. That said, I have hopes for a 65B 3-bit quantized alpaca-fine tuned. We'll see when some…

It's interesting that when ChatGPT 3.5 came out - everyone said, this is it! It's ready for primetime.

And now that there's a few competitors in the same league - 3.5 quality is suddenly garbage and only 4.0 is good enough.

Was it good enough before or wasn't it?

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#107

Post by simonw »

UncleOxidant wrote: post #95

Is it accurate to say they were trained for less than $600? Wouldn't that just be the finetuning that was done to the already existing LLaMA parameters which likely cost way more than $600 to train?

Yeah, exactly. LLaMA 7B itself cost $80,000+ to train (82,432 GPU hours). Stanford spent $100 on fine-tuning compute and $500 on OpenAI credits to generate their 52,000 sample instruction training set.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#108

Post by EGreg »

tantalor wrote: post #102
EGreg wrote: post #75

I warned about this for years. Finally an article gets it right. Everyone will soon have the equivalent of online nuclear weapons: bot swarms that infiltrate every forum, including this one.

Spam has existed on the internet for a long time.

This is different. It can act just like humans do for most people who skim comments won't be able to tell the difference.

Note this was in 2020: https://www.technologyreview.com/2020/10/08/1009845/a-gpt-3-...

And here's 4chan bot: https://www.youtube.com/watch?v=efPrtcLdcdM

I can tell you that HN is probably already being infiltrated as well.

SPAM can't gang up on you in a forum and downvote you and turn your friends against you and destroy your reputation within 1 hour online. But soon, it will. The web as we know it is soon going to be over.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#109

Post by bioemerl »

onlyrealcuzzo wrote: post #106
superkuh wrote: post #8

Hardly. I've played a lot with the 7,13, and 30B llamas as well as the 7 and 13B alpacas fine tuned by Stanford. They do not have emergent abilities like being able to generate rhymes or, say, represent a movie plot as emoji. Even openai's old text-davinci-003 (gpt3.5, but text completion, not the chat ones) far outperforms them. That said, I have hopes for a 65B 3-bit quantized alpaca-fine tuned. We'll see when some…

It's interesting that when ChatGPT 3.5 came out - everyone said, this is it! It's ready for primetime. And now that there's a few competitors in the same league - 3.5 quality is suddenly garbage and only 4.0 is good enough. Was it good enough before or wasn't it?

It's really not good enough yet, it's impressive for what it is in our current time. But we're looking at the 1980s computers.

They are neat, they are useful, but they can do so much more.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#110

Post by MildlySerious »

vdfs wrote: post #89
ianlevesque wrote: post #77

Earlier quoted context omitted.

That's ok, just ask ChatGPT to explain them to you.

I wonder if they will add "chatgpt it" or "gpt it" to the dictionaries

I wouldn't be surprised if GPT ends up as the the AskJeeves of LLMs.
Post reply on HN