Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

181–190 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#181

Post by sour-taste »

magicalhippo wrote: post #175
koheripbal wrote: post #151

Earlier quoted context omitted.

The difference is small, UNTIL you get to 4 bit quantization, where the model is noticeably dumber. 8 bits, imo, is the minimum.

Some parameters would be more sensitive than others I suppose? So could you use 4 bits for most, and 8 bits, or even 16, for the remaining?

I know nothing about this so my opinion means little, but I imagine it's hard to know which parameters are important enough to use more bits for.

I do wonder if it would be possible to have the model determine during training how important each parameter is, while maybe rewarding it for having more small parameters?

Re: Alpaca: A strong open-source instruction-following model

#182

Post by ronsor »

throwaway1851 wrote: post #170
clbrmbr wrote: post #7

Earlier quoted context omitted.

Just because they cannot be copyrighted doesn’t mean that they cannot be considered a trade secret. The agreement for using the product could done with an NDA. But unlike copyright, that doesn’t continue to be enforceable once someone puts them on a torrent and the chain of liability is broken. /notalawyereither

Well, trade secret law requires the owner of the trade secret to take steps to keep it... secret. Facebook has done the opposite, and they haven’t made the people they distributed the weights to sign an NDA.

And Facebook employees openly discuss the torrent magnet links.

Re: Alpaca: A strong open-source instruction-following model

#183

Post by stavros »

sebzim4500 wrote: post #139
SteveDR wrote: post #131

Earlier quoted context omitted.

I thought ChatGPT is only 20B parameters to begin with? (Source https://www.forbes.com/sites/forbestechcouncil/2023/02/17/is... )

I haven't seen anything official from OpenAI confirming that ChatGPT has fewer than 175B parameters, although it is a reasonable guess if you read between the lines of their statements. Given the author of that article is a CEO of an 'AI Ad Optimization Platform' I think that number is speculative at best.

ChatGPT is fine-tuned InstructGPT, which has 1.3B parameters, if I'm not mistaken.

Reference for the former: https://www.technologyreview.com/2023/03/03/1069311/inside-s...

Re: Alpaca: A strong open-source instruction-following model

#184

Post by antondd »

kossTKR wrote: post #149

A Tangent, but how long before we will see half the population having relationships with their AI assistants like in the sci-fi movie "Her". Add cutting edge voice synthesis and maybe a realistic video stream in a few years and we're there. Japan already has problems with the concept of "2d Waifus" and lack of intimacy between the younger generation. Maybe the downfall will not just be climate catastrophe but hyper i…

Once we have models that can generate perfect audio and video + better AR/VR tech, it’s game over.

Re: Alpaca: A strong open-source instruction-following model

#185

Post by hgsgm »

magicalhippo wrote: post #179
sebzim4500 wrote: post #142

Earlier quoted context omitted.

Having fewer than 1 bit per weight is not absurd. E.g. you can use 2 bits to represent 3 'weights' if you insist that at most one of the weights is allowed to exist. If you try to order nodes so that adjacent nodes are uncorrelated the performance loss might be manageable. People are already doing stuff like this (see sparsification) so it is conceivable to me that this is just what networks will look like in a few y…

> If you try to order nodes so that adjacent nodes are uncorrelated the performance loss might be manageable. shower thought In graphics we use barycentric coordinates to encode the position within an arbitrary triangle using two coordinates (u,v), with the third being constrained to be 1-u-v. If you order nodes to be correlated, could you use a similar trick to encode three weights for the price of two?

Yes, it's the same thing.

Re: Alpaca: A strong open-source instruction-following model

#186

Post by yellowapple »

atleastoptimal wrote: post #133
simonw wrote: post #28

This is why I think we're seeing a Stable Diffusion moment for LLMs: https://simonwillison.net/2023/Mar/11/llama/ Look at the timeline: 24th February 2023: LLaMA is announced, starts being shared with academic partners: https://research.facebook.com/publications/llama-open-and-ef... 2nd March: Someone posts a PR with a BitTorrent link to the models: https://github.com/facebookresearch/llama/pull/73 10th March: First…

Question: what percentage of the hype and momentum for this is so people can run sex chatbots on their local machine?

Feature-length AI-generated pornos don't seem that far off the horizon.

Re: Alpaca: A strong open-source instruction-following model

#188

Post by dongping »

magicalhippo wrote: post #175
koheripbal wrote: post #151

Earlier quoted context omitted.

The difference is small, UNTIL you get to 4 bit quantization, where the model is noticeably dumber. 8 bits, imo, is the minimum.

Some parameters would be more sensitive than others I suppose? So could you use 4 bits for most, and 8 bits, or even 16, for the remaining?

That's exactly why bitsandbytes has a threshold parameter to control the quantization.

Re: Alpaca: A strong open-source instruction-following model

#189

Post by hgsgm »

mx20 wrote: post #100
Animats wrote: post #45

Earlier quoted context omitted.

The US doesn't have database copyright, but some countries, including the entire European Union, do.[1] [1] https://en.wikipedia.org/wiki/Database_right

Interesting. That could be a huge disadvantage for the EU and it's companies that want to use AI and also often can't give their Data to third parties. But I am not sure a neural net is even considered a DB. Because when this was created they clearly had structured Information accessed by people in mind. There has to be some distinction otherwise I make a db that stores digits of PI in a file and if you publish part…

If I copy your database, that violates copyright. I generate the same data using my own process, it doesn't.

That's different from creative works where it's implausible to claim that you created the same work independently of someone else.

Re: Alpaca: A strong open-source instruction-following model

#190

Post by hgsgm »

Der_Einzige wrote: post #10
bryananderson wrote: post #4

Earlier quoted context omitted.

I am not a lawyer, but the consensus (around here anyway) seems to be that weights cannot be copyrighted since they are the output of an automated process, so perhaps the license cannot limit how the weights are used. The code can be copyrighted, but the code can also be reimplemented from scratch and used to load the weights.

God damn I hope this is where IP law lands. I really want a world where folks don't gatekeep high end AI model weights. Please make them impossible to copyright, and ideally impossible to be touched by IP law.

Why? I mean, do you feel differently about every other patented invention or IP? That would be weird.
Post reply on HN