Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alpaca: A strong open-source instruction-following model

crfm.stanford.edu

Post reply on HN
Print view View original

141–150 of 313 posts

Re: Alpaca: A strong open-source instruction-following model

#141

Post by generalizations »

bogwog wrote: post #31
adam_arthur wrote: post #8

Pretty exciting to see the democratization of this tech. The next wave of innovation seems like it’ll be much less centralized into a handful of big companies. Even more powerful closed door LLMs will appear, but already so much utility out of these that can run on consumer hardware.

Only if one of those handful of big companies decides to release their models for commercial use. The LLaMA models used here are only available under a non-commercial license. Can’t build anything significant with leaked models. Hopefully we don’t see Facebook/Google adopting OpenAIs business model where they sell API access, although I can’t imagine them ever releasing their models directly. The last thing they woul…

> Can’t build anything significant with leaked models.

If it's behind a SaaS, I think you can.

Re: Alpaca: A strong open-source instruction-following model

#142

Post by sebzim4500 »

londons_explore wrote: post #105
londons_explore wrote: post #87

Earlier quoted context omitted.

Yes. In a dense everything to everything neural network layer, the number of 'inputs' to a node is proportional to the square root of the number of weights. Therefore, assuming quantization noise is uncorrelated, as the number of weights doubles, the number of inputs goes up by sqrt(2), and the (normalized) noise goes down by a factor of 2*(sqrt(2)). So, as a rule of thumb, you can remove 1 bit of precision of the we…

Something is wrong with this math... by your logic I could scale the network up big enough that I could quantize the weights down to zero bits...

Having fewer than 1 bit per weight is not absurd. E.g. you can use 2 bits to represent 3 'weights' if you insist that at most one of the weights is allowed to exist. If you try to order nodes so that adjacent nodes are uncorrelated the performance loss might be manageable.

People are already doing stuff like this (see sparsification) so it is conceivable to me that this is just what networks will look like in a few years.

Re: Alpaca: A strong open-source instruction-following model

#143

Post by sebzim4500 »

atleastoptimal wrote: post #133
simonw wrote: post #28

This is why I think we're seeing a Stable Diffusion moment for LLMs: https://simonwillison.net/2023/Mar/11/llama/ Look at the timeline: 24th February 2023: LLaMA is announced, starts being shared with academic partners: https://research.facebook.com/publications/llama-open-and-ef... 2nd March: Someone posts a PR with a BitTorrent link to the models: https://github.com/facebookresearch/llama/pull/73 10th March: First…

Question: what percentage of the hype and momentum for this is so people can run sex chatbots on their local machine?

A lower portion than the equivalent number for Stable Diffusion, but still significant.

Re: Alpaca: A strong open-source instruction-following model

#145

Post by avmich »

With so much of texts being written by computers, can we have help on the reading side too? My emails are checked for spam and filtered, ok, but I'd still like to learn what are the important parts of the messages which pass through. Do we now have solutions for that already?

Re: Alpaca: A strong open-source instruction-following model

#146

Post by [deleted] »

gamegoblin wrote: post #62
2bitencryption wrote: post #46

Earlier quoted context omitted.

Here is one question I have not seen answered yet: All the magic of "7B LLaMA running on a potato" seems to involve lowering precision down to f16 and then further quantizing to int4. Clearly this quantized model still outputs something resembling human language, at the very least. But I haven't seen anyone show what effect this quantizing has on the quality of the output. If the quality of the output is bad, it's un…

Decrease in accuracy is negligible and decreases as model size increases. That is, larger models quantize even better than smaller models. https://arxiv.org/abs/2210.17323

[deleted]

Re: Alpaca: A strong open-source instruction-following model

#147

Post by simonw »

avmich wrote: post #145

With so much of texts being written by computers, can we have help on the reading side too? My emails are checked for spam and filtered, ok, but I'd still like to learn what are the important parts of the messages which pass through. Do we now have solutions for that already?

Language models like LLaMA (and GPT-3, ChatGPT etc) are really good at exactly this. They can summarize text, they can extract key points into bullets, they can figure out common topics and so on.

Re: Alpaca: A strong open-source instruction-following model

#149

Post by kossTKR »

A Tangent, but how long before we will see half the population having relationships with their AI assistants like in the sci-fi movie "Her".

Add cutting edge voice synthesis and maybe a realistic video stream in a few years and we're there.

Japan already has problems with the concept of "2d Waifus" and lack of intimacy between the younger generation.

Maybe the downfall will not just be climate catastrophe but hyper isolated people living alone with their ultra realistic bot friends and family without any desire to experience the ups and downs of actual social experience.

Extrapolate a bit and you wont know what's real outside of direct experience. Things are getting crazy!

Re: Alpaca: A strong open-source instruction-following model

#150

Post by LeoPanthera »

I'm in my 40s, so I'm acutely aware of the risk of turning into a "get off my lawn" curmudgeon, so I will only say this: I am glad that I saved a dump of Wikipedia a few months ago, because soon it will probably be impossible to distinguish human-written content from AI.
Post reply on HN