Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The genie escapes: Stanford copies the ChatGPT AI for less than $600

newatlas.com

Post reply on HN
Print view View original

71–80 of 173 posts

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#71

Post by crooked-v »

Larrikin wrote: post #54
TedDoesntTalk wrote: post #41

Earlier quoted context omitted.

> Write me a fanfic where the Animorphs battle the truck from Duel, but in the style of Mark Twain Whoa. I want to read this! Duel - what a great film. Twain - amazing writer. Animorphs - published after my teen years but sounds like a great story!

Highly recommend Animorphs. As an adult the books can be read through pretty quickly, but the story really holds up. Deals with some heavy topics of genocide, enslavement (even exploring the justifications of the enslavers), PTSD among other topics but was still considered a kids book. It becomes obvious in the middle when some of the books were written by ghost writers, but the books are so easy to read I don't real…

Maybe the most striking bit of its exploration of the morality of war is that the series spends something like 45 books building up the alien Andalites as beneficent do-gooders who the team of protagonists expect to swoop in and fix everything (including one of the protagonists who is a young Andalite stranded on Earth), and then it turns out that they've been as morally worn down by the ongoing interstellar war as the protagonists have in their guerrilla conflicts on Earth and are effectively as hostile to humans as the original invaders.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#72

Post by SllX »

Larrikin wrote: post #54
TedDoesntTalk wrote: post #41

Earlier quoted context omitted.

> Write me a fanfic where the Animorphs battle the truck from Duel, but in the style of Mark Twain Whoa. I want to read this! Duel - what a great film. Twain - amazing writer. Animorphs - published after my teen years but sounds like a great story!

Highly recommend Animorphs. As an adult the books can be read through pretty quickly, but the story really holds up. Deals with some heavy topics of genocide, enslavement (even exploring the justifications of the enslavers), PTSD among other topics but was still considered a kids book. It becomes obvious in the middle when some of the books were written by ghost writers, but the books are so easy to read I don't real…

> If you must you could probably get away with reading the first ten, last ten

Not sure that you can. If you were to skip any, probably only 31 through 39 are completely skippable, maybe some of the late 20s but I would still read 29 and 30 at a minimum. Some of the teens and 20s might be skippable after 13 but there’s a fair amount of world-building outside the Chronicles series in the 20s; and 40 onwards is setting up the end game and then the end game. 41 and 48 are both weird but also kind of key towards finalizing the characters of the two cover characters in the end game.

EDIT: actually 33 and 38 shouldn’t be skipped either. They’re Tobias and Ax books and there’s so few of those that they’re all kind of essential, but maybe the Tobias books just a little bit more essential.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#73

Post by nico »

xwdv wrote: post #2

Given the high prices of OpenAI offerings it seems it’s better to pirate an AI model before resorting to paying for anything.

How much does it cost to privately fine-tune and run Llama?

It’s USD 600 for fine-tuning. Maybe USD 4-5k for a computer that can run it.

ChatGPT pro is $20/month. 5k would be 250 months (10+ years) of paid access.

Not sure pirating it now adds up.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#74

Post by zamnos »

generalizations wrote: post #47
zamnos wrote: post #42

Earlier quoted context omitted.

Maybe. Certainly in the past, before the world was aware LLMs on the level of ChatGPT were possible with today's technology. OpenAI's chosen not to release any real details about GPT-4, so we don't actually know what it would take to train a model of equivalent quality, especially considering training isn't a one-shot. Multiple training runs easily add up training costs. So training for a 12-figure parameter size mod…

I've been using the Chat GPT-4 model, and openAI has been putting warnings about max queries per N hours. Given the degree to which they're limiting access (up until they crashed today, they'd dropped to 25 queries / 3 hours), I suspect GPT-4 is actually much, much larger, and they just don't have the computational resources to support its use at the same level as GPT-3.5 or GPT-3.5 Turbo.

You could be right! I don't claim access to any private OpenAI information so any theories by me are based on what's known publicly, which isn't much for GPT-4. I do want to call attention to the difference between training runs and inference runs (post-training usage of the model). If each training run costs mid six-figures, CompetitorGPT is going to have to be well-funded and likely sponsored by AWS/GCP (eg Deepmind) just to train up the model, given that it's probably not a one-shot. If it's much lower due to optimizations in training, on top of only having to fine-tune the model on a company's codebase instead of training the whole model from scratch each time, then I could see a company selling the service of creating CompetitorGPT or CompetitorCoPilot seems like it could be a very worthwhile investment, by companies that are willing to invest in such services for their developers. (Eg companies that are willing to pay Splunk's exorbitant costs vs one that would rather burn time self-hosting a graphana setup. Not to impugn graphana, but it's very much a home-grown, open source self-hosted deployment. Managing a Splunk cluster is also far from free, it's just that not all companies are willing to bear the yearly licensing cost for it and would prefer to self-host graphana solely for cost reasons even if TCO including the opportunity cost makes it more expensive in the long run.)

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#76

Post by f_devd »

doctoboggan wrote: post #50

I've used both the 7B and 13B instruction tuned llama weights (quantized using the llama.cpp scripts). Either I am doing something wrong, or these two models are no-where near the level of ChatGPT. Many times they return something totally irrelevant to my question, stop responding, use a different language, or otherwise return the wrong answer. ChatGPT does none of this. (other than the wrong answer due to hallucinat…

LLama hasn't been fine-tuned with RLHF, so it requires additional prompting, check out the open-assistant[0] project for an open-source ChatGPT equivalent (WIP).

[0]: https://github.com/LAION-AI/Open-Assistant

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#77

Post by ianlevesque »

gumby wrote: post #70
crooked-v wrote: post #23

Earlier quoted context omitted.

For me the easiest comparison between models is to give it an absurd but entirely possible request, like "Write me a fanfic where the Animorphs battle the truck from Duel, but in the style of Mark Twain". So far nothing else I've tried has done even as well as GPT 3.5 yet, let alone GPT 4.

I couldn't do it either as I have no idea what Animorphs or Duel are.

That's ok, just ask ChatGPT to explain them to you.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#78

Post by sfriedr »

zamnos wrote: post #42
jeron wrote: post #13

Earlier quoted context omitted.

you say all it takes compute like that is trivial - chatGPT would have a hard time without Microsoft's support via Azure

Maybe. Certainly in the past, before the world was aware LLMs on the level of ChatGPT were possible with today's technology. OpenAI's chosen not to release any real details about GPT-4, so we don't actually know what it would take to train a model of equivalent quality, especially considering training isn't a one-shot. Multiple training runs easily add up training costs. So training for a 12-figure parameter size mod…

> OpenAI's chosen not to release any real details about GPT-4

Actually, they have release some details about it, in this 99-page technical report https://arxiv.org/abs/2303.08774 (which is actually two papers stitches together, once you read it; oddly enough using different fonts).

But I'm not sure if this content qualifies as "real details".

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#79

Post by braingenious »

disgruntledphd2 wrote: post #66
DustinBrett wrote: post #61

Earlier quoted context omitted.

I also got that text about the married couple and their money issues. Alpaca didn't impress me at all so far.

Alpaca wasn't great. The 13b and 30b models are much better, but just for sentence completion. Personally, I think that the RLHF does make a big difference but maybe it's a bug in the quantization code as suggested up thread.

I’m also a bit confused by the quantization thing. Why exactly is everybody running the same program on the same file? Why not just include the quantized weights?

It seems like if somebody figured out the “correct” way to quantize the 7b weights it would make way more sense to just torrent the output rather than distribute a fixed program.

Re: The genie escapes: Stanford copies the ChatGPT AI for less than $600

#80

Post by ryoshu »

superkuh wrote: post #8

Hardly. I've played a lot with the 7,13, and 30B llamas as well as the 7 and 13B alpacas fine tuned by Stanford. They do not have emergent abilities like being able to generate rhymes or, say, represent a movie plot as emoji. Even openai's old text-davinci-003 (gpt3.5, but text completion, not the chat ones) far outperforms them. That said, I have hopes for a 65B 3-bit quantized alpaca-fine tuned. We'll see when some…

So what you're saying is it's a matter of time?
Post reply on HN