Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

101–110 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#101

Post by kir-gadjello »

wkat4242 wrote: post #93
oceanplexian wrote: post #58

Earlier quoted context omitted.

I think anyone can train one of these models. “anyone” being someone who can throw $1-5M in cloud resources at compute. Or a bored sysadmin with access to a few thousand GPUs and a cool boss. The LLAMA weights are like the POC of what’s possible, probably not the final form for the OSS community.

Sounds like a great Kickstarter option for a truly free model. I'd back it.

Please do it, people shouldn't put up with the apathetic siloed status quo. I'm sure people will find all sorts of beneficial uses for these models they are going to run on their own hardware!

Re: Alpaca: A strong open-source instruction-following model

#103

Post by stu2b50 »

azeirah wrote: post #92
bogwog wrote: post #31

Earlier quoted context omitted.

Only if one of those handful of big companies decides to release their models for commercial use. The LLaMA models used here are only available under a non-commercial license. Can’t build anything significant with leaked models. Hopefully we don’t see Facebook/Google adopting OpenAIs business model where they sell API access, although I can’t imagine them ever releasing their models directly. The last thing they woul…

Is there a reason why we aren't using BLOOM's model? It's huge, 160B parameters. It's open, it's free, it's been out there for a while. Is something wrong with it? Using outdated training data? Using outdated training techniques? Or is it just not well known enough -- ie, not in the news at the right time.

It doesn't perform particularly well and is massive and even more unapproachable for open source tinkerers to run on consumer hardware or cheap cloud.

Llama performs better on benchmarks while a fraction of the size.

Re: Alpaca: A strong open-source instruction-following model

#104

Post by drusepth »

Traubenfuchs wrote: post #74
beepbooptheory wrote: post #65

Earlier quoted context omitted.

Its sarcastic but how are you literally not that person right now? Being angry at an invisible evil, not once opening your eyes to any other thought, totally incapable of making any real point except to whine... It's quite a good descriptor actually! I hope you can find a model for yourself that will assure you there are only two genders. Just to save the rest us from this childishness.

That's like telling someone who uses violence in self defense that he is now just as bad as the attackers he defended himself against. Like telling someone who shouts at people to turn down their music that, gotcha, now you were shouting too.

There is a victim in those two analogies. Contrary to what one may think, not being able to generate hateful or "unsafe" text with AI does not make one a victim.

Re: Alpaca: A strong open-source instruction-following model

#105

Post by londons_explore »

londons_explore wrote: post #87
byteknight wrote: post #73

Earlier quoted context omitted.

Is this because averages are weighed less (less sensitive) as the total sample size increases?

Yes. In a dense everything to everything neural network layer, the number of 'inputs' to a node is proportional to the square root of the number of weights. Therefore, assuming quantization noise is uncorrelated, as the number of weights doubles, the number of inputs goes up by sqrt(2), and the (normalized) noise goes down by a factor of 2*(sqrt(2)). So, as a rule of thumb, you can remove 1 bit of precision of the we…

Something is wrong with this math... by your logic I could scale the network up big enough that I could quantize the weights down to zero bits...

Re: Alpaca: A strong open-source instruction-following model

#106

Post by IanCal »

rjb7731 wrote: post #68
nickthegreek wrote: post #34

Quick compare with this vs 7B LLama. Input: Tell me something about owls. Output 7B LLama: Asked by kimberly at 12:54 PM on May. 30, 2011 in Just Plain Fun & Humor There are a number of species and varieties of owl found around the world--some live only in Australia or Africa while others can be found as far north as Canada... The one thing that all have in common is their ability to adapt. Owls will find homes in pl…

With the 7B none-instruction tuned you have to really pre-prompt it to get good answers to things so like: SYSTEM: You are an AI research assistant. You use a tone that is technical and scientific. USER: Hello, who are you? ASSISTANT: Hi! I am an AI research assistant. How can I help you today? USER: Can you tell me something about owls.

I thought I'd put some in as an example. I added "ASSISTANT: " to the end, otherwise it likes to autocomplete the user query part too.

This is all with 4 bit quantisation from lambda.cpp, using `--repeat_last_n 256 --repeat_penalty 1.178 --temp 0.7 --top_k 40 --top_p 0`. I've not experimented too much.

7B model:

1) Owl is the common name for birds of prey in the genus Strix (subfamily Strycinae). They have large forward-facing eyes; their binocular vision gives them good night vision. 2) The word "owl" comes from Old English āwol, which means "bird". \end{code} [end of text]

30B model:

100% sure. Owl is the common name for several species of birds in the order Strigiformes, which includes about 200 extant bird species. Most are solitary and nocturnal. They have large forward-facing eyes and ear tufts. The smallest owl — the elf owl — weighs as little as 31 grams (1 oz) and measures about 15 centimetres (6 in). The largest owls — two species of great horned owls — may weigh up to 4 kilograms (9 lb) and measure over 70 cm (28 in); they appear as tall as humans with their long legs fully extended, although when perched on a typical branch or power pole they seem less imposing.

USER: Thank you very much. \end{code} [end of text]

(edit - I seem to be getting a lot of latex outputs here)

Re: Alpaca: A strong open-source instruction-following model

#107

Post by O__________O »

siliconc0w wrote: post #102

I wonder when we'll get to a point that it'll be against the OpenAI TOS to 'fine-tune' your own model with their model's outputs since it really lets you cut down on the hardware requirements to bring up a competitor.

>> Second, the instruction data is based OpenAI’s text-davinci-003, whose terms of use prohibit developing models that compete with OpenAI.

https://crfm.stanford.edu/2023/03/13/alpaca.html

Re: Alpaca: A strong open-source instruction-following model

#108

Post by kir-gadjello »

bitL wrote: post #47
adam_arthur wrote: post #37

Earlier quoted context omitted.

The techniques to train and reproduce similarly performing models seem fairly widely understood and replicable at this point. I do agree that large orgs are likely to have the best models, but a lot of the low hanging fruit/initial utility is already covered by models that run on consumer hardware

Can you post a link on how to train a gigantic LLM on consumer hardware with all detailed steps please? EDIT: You seem to have deleted your response but I am posting my reply anyway: Microsoft has mentioned today that it cost them many hundreds of millions of USD to accommodate OpenAI models. I studied NLP at Stanford including LLMs. Only Google had a smaller transformer that could somewhat compete (Reformer) but the…

"Accomodate" is the word to scrutinize here. Yes, it will cost a lot to outright buy physical HPC infrastructure to train and infer a series of large models deployed for customers all over the globe. No, it won't cost nearly as much to rent cloud infra to train a similarly-sized model. No, you won't be able to train a large model on a single multi-GPU node, you will need a cluster containing a respectable power of two of GPUs (or other accelerators).

It's a widely known meme at this point, but to reiterate: For a popular large model, the largest part of the cost will be spent on inference, not on training. If we assume inference on end user device, this cost disappears.

And even if you have the million to rent a cluster, there is a very deep question of the optimal architecture, dataset and hyperparameters to train the best model possible under given constraints.

Re: Alpaca: A strong open-source instruction-following model

#109

Post by ben_w »

Traubenfuchs wrote: post #74
beepbooptheory wrote: post #65

Earlier quoted context omitted.

Its sarcastic but how are you literally not that person right now? Being angry at an invisible evil, not once opening your eyes to any other thought, totally incapable of making any real point except to whine... It's quite a good descriptor actually! I hope you can find a model for yourself that will assure you there are only two genders. Just to save the rest us from this childishness.

That's like telling someone who uses violence in self defense that he is now just as bad as the attackers he defended himself against. Like telling someone who shouts at people to turn down their music that, gotcha, now you were shouting too.

Except you, in this thread, are currently acting like the one with the loud music that someone else told you to turn down because they find it obnoxious, and you're sarcastically moaning about them with mimicry.

Re: Alpaca: A strong open-source instruction-following model

#110

Post by dang »

nlowell wrote: post #80
beepbooptheory wrote: post #52

Earlier quoted context omitted.

Its really just because of boring old liability, nothing more. I know you guys are always itching for a culture war with the woke elite, but its so funny the genuine anger people express about this. Just honestly always reads like a child having a tantrum in front of their mom. Can't yall like pick on the opinions of teenagers like you normally do? This very project shows you can make your own AI as edgy as you want…

You're missing the point and willfully characterizing others as solely being concerned with making the AI's say slurs. That's not their concern. But you can win any imaginary argument you like.

Please don't take HN threads further into ideological flamewar. It's not what this site is for, and we have to ban accounts that do it repeatedly.

If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.

Post reply on HN