State-of-the-art open-source chatbot, Vicuna-13B, just released model weights • Hacker News Board

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#141

Post by muyuu » Fri, Apr 07, 2023, 9:54 PM UTC

andai wrote: post #140

muyuu wrote: post #138

Earlier quoted context omitted.

btw I got 4bit quantized Vicuna working in my 16GB laptop and the results seem very good, perhaps the best I got running locally so far

Did you have to diff LLaMA? Did you use EasyLM?

I found it ready-made for download, here https://huggingface.co/eachadea/ggml-vicuna-13b-4bit

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#142

Post by ThorsBane » Sat, Apr 08, 2023, 11:44 PM UTC

pugio wrote: post #126

d4rkp4ttern wrote: post #122

I have an M1 MBP 64GB. Can I run it on my M1 Or do I need a GPU ?

I have it running, slowly, on the same machine. I would love for someone to get support running for MPS backend (The GPU) but it does run on the CPU.

Thanks for sharing. How slowly is slowly? Do you anticipate that an M2 Max with 96gb of memory would run it noticeably faster?

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#143

Post by dc443 » Sun, Apr 09, 2023, 5:43 PM UTC

stevenhuang wrote: post #109

gigel82 wrote: post #95

One of my favorite queries to assess an LLM is Is it legal for a man to marry his widow's sister? Vicuna-13B: The legality of a man marrying his widow's sister (widow marriage) varies depending on the country and cultural context. In some societies, widow marriage was traditionally practiced or tolerated. GPT-4: It is not possible for a man to marry his widow's sister, as the term "widow" refers to a deceased man's w…

Nice test, cool to see gpt4 got it. You'd probably need to come up with a new one now though, or confirm knowledge cutoff for the next evaluation :p

Ouch. I got this wrong and was under the impression that GPT4 got this wrong for half an hour, and then figured out after reading it again after returning from a walk that this is one hell of a trick question. My brain automatically assumed that a man's widow is the man's dead wife, but I see that the correct way to interpret this is to realize that it means the man is the one who is dead.

It's pretty awesome to realize that from now onward my computers are going to be able to help catch more and more of the holes that clearly exist in my cognition.