Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

twitter.com

Post reply on HN
Print view View original

111–120 of 143 posts

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#111

Post by kybernetikos »

logicchains wrote: post #104
kybernetikos wrote: post #98

Earlier quoted context omitted.

If NN weights aren't protected by IP law that could slow down progress quite a lot. That could be very good for people worried about alignment.

>If NN weights aren't protected by IP law that could slow down progress quite a lot. What do you mean? IP law is overwhelmingly an impediment to progress; innovation happens faster when people are free to build on existing weights.

Yes, but there's less incentive for large companies to spend huge amounts of money training these systems when other companies can just take their work for free.

Removing IP protection would make it a lot easier to innovate at this level, but it would reduce the amount of money flowing into getting us to the next level.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#112

Post by swader999 »

EVa5I7bHFq9mnYK wrote: post #101
dragonwriter wrote: post #52

Earlier quoted context omitted.

> a massive search operation, akin to finding the right temperature to bake a cake from all possible floats ...for each of 13 billion (for a model with that many parameters) different cakes, except that they aren’t like cakes because the “best" temperature for each depends on the actual temperatures chosen for the others.

It's 2^(16*13,000,000,000) different cakes.

Way better than paperclips.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#113

Post by muyuu »

ode wrote: post #31

Is there some single page that keeps a running status of the various LLVM's and the software to make them runnable on consumer hardware?

consumer hardware is a bit vague of a limitation, which I guess it's partly why people are not tracking precisely what runs on what very closely

these could be useful:

https://nixified.ai

https://github.com/Crataco/ai-guide/blob/main/guide/models.m... -> https://old.reddit.com/user/Crataco/comments/zuowi9/opensour...

https://github.com/cocktailpeanut/dalai

the 4-bit quantized version of LLaMA 13B runs on my laptop without a dedicated GPU and I guess the same would apply to quantized vicuna 13B but I haven't tried that yet (converted as in this link but for 13B instead of 7B https://github.com/ggerganov/llama.cpp#usage )

GPT4All Lora's also works, perhaps the most compelling results I've got yet in my local computer - I have to try quantized Vicuna to see how that one goes, but processing the files to get a 4bit quantized version will take many hours so I'm a bit hesitant

PS: converting 13B Llama took my laptop's i7 around 20 hours and required a large swap file on top of its 16GB of RAM

feel free to answer back if you're trying any of these things this week (later I might lose track)

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#114

Post by atemerev »

This model is also censored to the brim, it refuses to answer half of my questions, some of them perfectly legal. It’s useless, we already have GPT-4 (and Vicuna is even more censored/guarded).

Alpaca-30B is much better, it will even tell you how to build a nuclear weapon (incorrectly, of course, it’s not that smart).

I am waiting for Coati13B weights, these should work great.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#115

Post by amrb »

anonymousgpt1 wrote: post #48
sillysaurusx wrote: post #38

Earlier quoted context omitted.

(I work on llama-dl.) We’re fighting back against the DMCA requests on the basis that NN weights aren’t copyrightable. This thread has details: https://news.ycombinator.com/item?id=35393782 I don't think you have to worry about Facebook going after you. The worst that will happen is that they issue a DMCA, in which case your project gets knocked offline. I don’t think they’ll be going the RIAA route of suing individu…

Lemme save whoever is donating the legal here the time: model weights are definitely copyrightable.

thanks zero comment bot account!

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#116

Post by pmoriarty »

kybernetikos wrote: post #111
logicchains wrote: post #104

Earlier quoted context omitted.

>If NN weights aren't protected by IP law that could slow down progress quite a lot. What do you mean? IP law is overwhelmingly an impediment to progress; innovation happens faster when people are free to build on existing weights.

Yes, but there's less incentive for large companies to spend huge amounts of money training these systems when other companies can just take their work for free. Removing IP protection would make it a lot easier to innovate at this level, but it would reduce the amount of money flowing into getting us to the next level.

Or development could shift out of the hands of these large corporations, which might be a good thing.

Somehow, though, I doubt they'll let the golden goose slip through their fingers, no matter what happens.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#117

Post by wkat4242 »

kybernetikos wrote: post #111
logicchains wrote: post #104

Earlier quoted context omitted.

>If NN weights aren't protected by IP law that could slow down progress quite a lot. What do you mean? IP law is overwhelmingly an impediment to progress; innovation happens faster when people are free to build on existing weights.

Yes, but there's less incentive for large companies to spend huge amounts of money training these systems when other companies can just take their work for free. Removing IP protection would make it a lot easier to innovate at this level, but it would reduce the amount of money flowing into getting us to the next level.

Not really. This model only made it to the public because meta was offering it publicly.

This won't happen to GPT any time soon so they are safe, copyright or not.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#118

Post by wkat4242 »

7to2 wrote: post #107
gigel82 wrote: post #66

All of the llama derivatives are tainted by Meta's license, which makes commercial use illegal and even personal use dubious. Is all the training material used for Llama available as open source? Maybe lots of folks can pool their resources and create fully open clean models / weights instead.

> All of the llama derivatives are tainted by Meta's license, which makes commercial use illegal and even personal use dubious. This is not true if you never agreed to Meta's license. If you haven't, you either can't redistribute the weights or you're completely free to use them as you see fit depending on whether weights are copyrightable (very likely) or not. We'll have to wait for the llama-dl lawsuit to find out…

For personal use, who cares? Really.. How would they even know I'm using it?

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#119

Post by wongarsu »

muyuu wrote: post #105
gaogao wrote: post #33

Earlier quoted context omitted.

It's fairly similar to a ROM patch in the video game space, which has mostly stood the test of time.

like that but requiring 60GBs of CPU RAM for some reason :-P one has to wonder how did they implement the storage of those deltas to require that sort of RAM

For perspective, that's about $200-$250 of RAM on a desktop computer. They might just not have cared.

Though I expect somebody to write a patch to make this more accessible to people on laptops.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#120

Post by wongarsu »

muyuu wrote: post #103
gigel82 wrote: post #95

One of my favorite queries to assess an LLM is Is it legal for a man to marry his widow's sister? Vicuna-13B: The legality of a man marrying his widow's sister (widow marriage) varies depending on the country and cultural context. In some societies, widow marriage was traditionally practiced or tolerated. GPT-4: It is not possible for a man to marry his widow's sister, as the term "widow" refers to a deceased man's w…

would still be possibly legal on the basis that if it's not illegal then it's legal - in British jurisprudence tradition at least https://en.wikipedia.org/wiki/Everything_which_is_not_forbid... - namely it's not law that impedes it (also in some places there's posthumous marriage)

There are also people who are considered dead by the bureaucratic system, but physically alive. Usually because of clerical errors that are sometimes surprisingly hard to resolve. In this context the wife of the man would be considered a widow in many contexts, despite her man being alive.
Post reply on HN