Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

171–180 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#171

Post by CuriouslyC »

nwoli wrote: post #36
simonw wrote: post #28

This is why I think we're seeing a Stable Diffusion moment for LLMs: https://simonwillison.net/2023/Mar/11/llama/ Look at the timeline: 24th February 2023: LLaMA is announced, starts being shared with academic partners: https://research.facebook.com/publications/llama-open-and-ef... 2nd March: Someone posts a PR with a BitTorrent link to the models: https://github.com/facebookresearch/llama/pull/73 10th March: First…

A lot of them aren’t very good though at the same stable diffusion vram level unfortunately (and we’ve had large non consumer level gpu llms open sourced for a while eg gpt-j)

A lot of people are running Llama using the CPU/system memory.

Re: Alpaca: A strong open-source instruction-following model

#172

Post by pmoriarty »

LeoPanthera wrote: post #150

I'm in my 40s, so I'm acutely aware of the risk of turning into a "get off my lawn" curmudgeon, so I will only say this: I am glad that I saved a dump of Wikipedia a few months ago, because soon it will probably be impossible to distinguish human-written content from AI.

If it's indistinguishable, then how does it matter who/what it was written by?

Re: Alpaca: A strong open-source instruction-following model

#174

Post by ntonozzi »

dmw_ng wrote: post #72
2bitencryption wrote: post #46

Earlier quoted context omitted.

Here is one question I have not seen answered yet: All the magic of "7B LLaMA running on a potato" seems to involve lowering precision down to f16 and then further quantizing to int4. Clearly this quantized model still outputs something resembling human language, at the very least. But I haven't seen anyone show what effect this quantizing has on the quality of the output. If the quality of the output is bad, it's un…

For 10 billion+ parameter models, the effects of quantization are relatively small, for smaller models like Llama 7B the effect becomes more dramatic, but there is ongoing research on new quantization methods (like GPTQ) that preserve significant performance even on the lower end. Quantization isn't the only technique available for downsizing a model, Llama itself is already the result of sizing the model and input d…

Llama is trained with _more_ data than is chinchilla optimal in order to make it better and cheaper at inference time, instead of just getting the highest quality of model that you can based on a given training budget. Llama has fewer parameters and was trained on more data specifically so that it would get high quality results on cheaper hardware and be easier and faster to run at inference time.

Re: Alpaca: A strong open-source instruction-following model

#175

Post by magicalhippo »

koheripbal wrote: post #151
dmw_ng wrote: post #72

Earlier quoted context omitted.

For 10 billion+ parameter models, the effects of quantization are relatively small, for smaller models like Llama 7B the effect becomes more dramatic, but there is ongoing research on new quantization methods (like GPTQ) that preserve significant performance even on the lower end. Quantization isn't the only technique available for downsizing a model, Llama itself is already the result of sizing the model and input d…

The difference is small, UNTIL you get to 4 bit quantization, where the model is noticeably dumber. 8 bits, imo, is the minimum.

Some parameters would be more sensitive than others I suppose? So could you use 4 bits for most, and 8 bits, or even 16, for the remaining?

Re: Alpaca: A strong open-source instruction-following model

#176

Post by sanxiyn »

whimsicalism wrote: post #156
famouswaffles wrote: post #3

https://crfm.stanford.edu/2023/03/13/alpaca.html They say they'll release the weights. But Of course, you'll be beholden to a non-commercial license. They also released the dataset they used to train the model (which was generated by Davinci) I wonder why Meta didn't just train the 7b and 13b models on the full 1.4T tokens. They curves show they weren't even close to converging. Shame really.

I am curious how you are expected to prove in a court of law that the weights you have are derived from some licensed weights if I take just a few gradient steps from the original weights.

Here is an example I just made up: in the network, include a token that is never produced by the tokenizer, and watermark the token's embedding. It won't be touched by normal fine tuning and will remain the same.

I am sure there are better methods.

Re: Alpaca: A strong open-source instruction-following model

#177

Post by earleybird »

kossTKR wrote: post #149

A Tangent, but how long before we will see half the population having relationships with their AI assistants like in the sci-fi movie "Her". Add cutting edge voice synthesis and maybe a realistic video stream in a few years and we're there. Japan already has problems with the concept of "2d Waifus" and lack of intimacy between the younger generation. Maybe the downfall will not just be climate catastrophe but hyper i…

To add to sci-fi examples, Surrogates https://www.imdb.com/title/tt0986263/

Re: Alpaca: A strong open-source instruction-following model

#178

Post by yieldcrv »

wkat4242 wrote: post #82
nickthegreek wrote: post #14

>We are waiting for huggingface to officially support the llama models (i.e. this PR to be merged) before we release a stable version of the finetuning code. Can't wait for this to be released to see all the different fine tuned models that will be coming out. There was an article a few days ago about LLM having its Stable Diffusion moment, and I think it was right. We went from LLama release > llama.cpp > Dalai and…

Yeah this whole industry seems to suddenly have moved from the pace of a pebble creek to a waterfall. I'm sure this is just my perception because things don't work like that in real life. Is it really just a ton of slowly developed things happening to mature close together?

ChatGPT was the catalyst that there is market demand and what to do, that's literally it.

Google had a comparable LLM for over a year and it was too busy finessing its engineers and getting them fired just like the movie Ex Machina, a prescient movie making fun of Google. The organization just sat on it. Plenty of others were in the same place, no imagined product market fit.

Re: Alpaca: A strong open-source instruction-following model

#179

Post by magicalhippo »

sebzim4500 wrote: post #142
londons_explore wrote: post #105

Earlier quoted context omitted.

Something is wrong with this math... by your logic I could scale the network up big enough that I could quantize the weights down to zero bits...

Having fewer than 1 bit per weight is not absurd. E.g. you can use 2 bits to represent 3 'weights' if you insist that at most one of the weights is allowed to exist. If you try to order nodes so that adjacent nodes are uncorrelated the performance loss might be manageable. People are already doing stuff like this (see sparsification) so it is conceivable to me that this is just what networks will look like in a few y…

> If you try to order nodes so that adjacent nodes are uncorrelated the performance loss might be manageable.

shower thought

In graphics we use barycentric coordinates to encode the position within an arbitrary triangle using two coordinates (u,v), with the third being constrained to be 1-u-v. If you order nodes to be correlated, could you use a similar trick to encode three weights for the price of two?

Re: Alpaca: A strong open-source instruction-following model

#180

Post by uh_uh »

z3t4 wrote: post #91
londons_explore wrote: post #23

The training data doesn't seem to be great quality... "instruction": "Calculate the sum of each column in the following table.", "input": "2 | 3 | 4\n 5 | 6 | 7", "output": "2+3+4 = 9\n5+6+7 = 18" I think better results could be obtained by manually going through these 52,000 training examples - even a couple of seconds per example should be enough to weed out obviously wrong ones, and should only take <$1000 of huma…

Training a model to do math is impossible. If you tell a child that 1+1=2 10+10=20 100+100=200 an "AI" can not figure out that 1000+1000=2000 like a child can.

Has this actually been tried? It would be trivial to generate a very large dataset of simple additions and see if an LLM gets it. Maybe it wouldn't serve an immediate practical purpose but I'm curious.
Post reply on HN