Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

151–160 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#151

Post by koheripbal »

dmw_ng wrote: post #72
2bitencryption wrote: post #46

Earlier quoted context omitted.

Here is one question I have not seen answered yet: All the magic of "7B LLaMA running on a potato" seems to involve lowering precision down to f16 and then further quantizing to int4. Clearly this quantized model still outputs something resembling human language, at the very least. But I haven't seen anyone show what effect this quantizing has on the quality of the output. If the quality of the output is bad, it's un…

For 10 billion+ parameter models, the effects of quantization are relatively small, for smaller models like Llama 7B the effect becomes more dramatic, but there is ongoing research on new quantization methods (like GPTQ) that preserve significant performance even on the lower end. Quantization isn't the only technique available for downsizing a model, Llama itself is already the result of sizing the model and input d…

The difference is small, UNTIL you get to 4 bit quantization, where the model is noticeably dumber.

8 bits, imo, is the minimum.

Re: Alpaca: A strong open-source instruction-following model

#152

Post by scottlawson »

redox99 wrote: post #29

They used OpenAI davinci to generate the dataset for instruction tuning. They mention one of the reasons this model is noncommercial is because OpenAI's terms of use prohibit developing models that compete with OpenAI. They also published the generated dataset. If one were to use this dataset to tune your own commercial model, would this be OK, or illegal? I would guess OpenAIs ToS no longer applies, as this is now A…

This model is for non commercial use only so it's unclear to me how it could compete commercially with OpenAI. Sure, you could decide to interpret "compete" in a way that makes it seem like this violates TOS, but the mom commercial restriction is a powerful counterargument too.

Re: Alpaca: A strong open-source instruction-following model

#153

Post by karmasimida »

bryananderson wrote: post #4
famouswaffles wrote: post #3

https://crfm.stanford.edu/2023/03/13/alpaca.html They say they'll release the weights. But Of course, you'll be beholden to a non-commercial license. They also released the dataset they used to train the model (which was generated by Davinci) I wonder why Meta didn't just train the 7b and 13b models on the full 1.4T tokens. They curves show they weren't even close to converging. Shame really.

I am not a lawyer, but the consensus (around here anyway) seems to be that weights cannot be copyrighted since they are the output of an automated process, so perhaps the license cannot limit how the weights are used. The code can be copyrighted, but the code can also be reimplemented from scratch and used to load the weights.

Not copy right, but doesn't mean this isn't someone's property.

Re: Alpaca: A strong open-source instruction-following model

#154

Post by rjb7731 »

Seems not too bad, you can tell its been trained on shorter responses.

Example #1

Input: explain a neural network in simple terms

Alpaca-Output: A neural network is a type of machine learning algorithm that is made up of layers of interconnected nodes (aka neurons). It is inspired by the way the brain works and uses interconnected nodes to learn patterns in data and make predictions. Neural networks can be used to perform various tasks such as classification, regression, and speech/image recognition.

Example #2

Input: explain how transformers work in artificial intelligence

Alpaca-Output: Transformers are a type of artificial intelligence algorithm used in deep learning. They are responsible for taking input data and converting it into a different data representation, called the "output space". Transformers do this by using a series of mathematical operations, allowing them to map input data onto a new, desired output. This allows the artificial intelligence system to perform tasks such as natural language processing and image recognition. Transformers are able to learn from data and can be used to build more efficient models for a variety of tasks

Example #3

Input: how do i make a request to the chatgpt api with python?

Alpaca-Output: The API can be accessed using the Python language. You can use requests library to make API requests, for example:

import requests

# URL of the chatgpt API url = 'https://api.chatgpt.com/v1/messages'

# Post a new message data = { 'text': 'Hello World' }

# Send the request and get the response response = requests.post(url, data=data) print(response.text)

Re: Alpaca: A strong open-source instruction-following model

#155

Post by whimsicalism »

wkat4242 wrote: post #82
nickthegreek wrote: post #14

>We are waiting for huggingface to officially support the llama models (i.e. this PR to be merged) before we release a stable version of the finetuning code. Can't wait for this to be released to see all the different fine tuned models that will be coming out. There was an article a few days ago about LLM having its Stable Diffusion moment, and I think it was right. We went from LLama release > llama.cpp > Dalai and…

Yeah this whole industry seems to suddenly have moved from the pace of a pebble creek to a waterfall. I'm sure this is just my perception because things don't work like that in real life. Is it really just a ton of slowly developed things happening to mature close together?

> Is it really just a ton of slowly developed things happening to mature close together

Yes. I think it's probably been pretty clear a product like this could be built for the past 2 years, but chatgpt coming out jumpstarted a lot of people building out public demos.

Re: Alpaca: A strong open-source instruction-following model

#156

Post by whimsicalism »

famouswaffles wrote: post #3

https://crfm.stanford.edu/2023/03/13/alpaca.html They say they'll release the weights. But Of course, you'll be beholden to a non-commercial license. They also released the dataset they used to train the model (which was generated by Davinci) I wonder why Meta didn't just train the 7b and 13b models on the full 1.4T tokens. They curves show they weren't even close to converging. Shame really.

I am curious how you are expected to prove in a court of law that the weights you have are derived from some licensed weights if I take just a few gradient steps from the original weights.

Re: Alpaca: A strong open-source instruction-following model

#157

Post by rkwasny »

z3t4 wrote: post #91
londons_explore wrote: post #23

The training data doesn't seem to be great quality... "instruction": "Calculate the sum of each column in the following table.", "input": "2 | 3 | 4\n 5 | 6 | 7", "output": "2+3+4 = 9\n5+6+7 = 18" I think better results could be obtained by manually going through these 52,000 training examples - even a couple of seconds per example should be enough to weed out obviously wrong ones, and should only take <$1000 of huma…

Training a model to do math is impossible. If you tell a child that 1+1=2 10+10=20 100+100=200 an "AI" can not figure out that 1000+1000=2000 like a child can.

Of course it is possible for LLM model to learn this!

The problem was with token embeddings, not with math.

Re: Alpaca: A strong open-source instruction-following model

#158

Post by dchest »

clbrmbr wrote: post #7
bryananderson wrote: post #4

Earlier quoted context omitted.

I am not a lawyer, but the consensus (around here anyway) seems to be that weights cannot be copyrighted since they are the output of an automated process, so perhaps the license cannot limit how the weights are used. The code can be copyrighted, but the code can also be reimplemented from scratch and used to load the weights.

Just because they cannot be copyrighted doesn’t mean that they cannot be considered a trade secret. The agreement for using the product could done with an NDA. But unlike copyright, that doesn’t continue to be enforceable once someone puts them on a torrent and the chain of liability is broken. /notalawyereither

Yeah, this happened with RC4 cipher which used to be a trade secret until someone posted it on Usenet: https://en.wikipedia.org/wiki/RC4#History

Re: Alpaca: A strong open-source instruction-following model

#159

Post by nestorD »

wkat4242 wrote: post #82
nickthegreek wrote: post #14

>We are waiting for huggingface to officially support the llama models (i.e. this PR to be merged) before we release a stable version of the finetuning code. Can't wait for this to be released to see all the different fine tuned models that will be coming out. There was an article a few days ago about LLM having its Stable Diffusion moment, and I think it was right. We went from LLama release > llama.cpp > Dalai and…

Yeah this whole industry seems to suddenly have moved from the pace of a pebble creek to a waterfall. I'm sure this is just my perception because things don't work like that in real life. Is it really just a ton of slowly developed things happening to mature close together?

> Is it really just a ton of slowly developed things happening to mature close together?

It is a bit of both. Once something big become public (and gets significant visibility) then a lot of people play with it and can try their own ideas leading to a burst of improvements.

Re: Alpaca: A strong open-source instruction-following model

#160

Post by simonw »

kossTKR wrote: post #149

A Tangent, but how long before we will see half the population having relationships with their AI assistants like in the sci-fi movie "Her". Add cutting edge voice synthesis and maybe a realistic video stream in a few years and we're there. Japan already has problems with the concept of "2d Waifus" and lack of intimacy between the younger generation. Maybe the downfall will not just be climate catastrophe but hyper i…

That's been happening already with Replika: https://www.abc.net.au/news/science/2023-03-01/replika-users...
Post reply on HN