Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

twitter.com

Post reply on HN
Print view View original

121–130 of 143 posts

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#121

Post by wongarsu »

capableweb wrote: post #28
simse wrote: post #25

It's actually very impressive. I gave it the task of converting a query and an OpenAPI spec into an API call, and it worked! I've not been succesful in getting GPT-3.5 to do this without rambling on about the reasoning for its decision.

Usually if I want code from the GPT family I always add "Just show me the code, no extra words or explanation" in the end of the prompt, and it works 99% of the time. Edit: just finished the conversion of Vicuna myself now and been doing some light testing, seems to work in ~80% of the cases for it, not as high success-rate as with GPT for sure. Probably there is a better way of structuring the prompt for Vicuna.

I wonder if that makes the code worse on average, considering how LLMs seem to dramatically increase their reasoning capabilities if you ask them to show their thought process

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#123

Post by moritzwarhier »

muyuu wrote: post #103
gigel82 wrote: post #95

One of my favorite queries to assess an LLM is Is it legal for a man to marry his widow's sister? Vicuna-13B: The legality of a man marrying his widow's sister (widow marriage) varies depending on the country and cultural context. In some societies, widow marriage was traditionally practiced or tolerated. GPT-4: It is not possible for a man to marry his widow's sister, as the term "widow" refers to a deceased man's w…

would still be possibly legal on the basis that if it's not illegal then it's legal - in British jurisprudence tradition at least https://en.wikipedia.org/wiki/Everything_which_is_not_forbid... - namely it's not law that impedes it (also in some places there's posthumous marriage)

Even that charitable interpretation doesn't help much when Vicuna hallucinates the > (widow marriage)

as if it were a common term.

Doesn't make Vicuna less impressive, it comes pretty close to Chat-GPT in many regards. And I like that trick question.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#124

Post by xbmcuser »

yreg wrote: post #106
xbmcuser wrote: post #88

Just when you think Nvidia will go down something happens that changes it. These days unless you were into gaming or a machine learning dev the integrated graphics were good enough. But now first time in a long time I am interested in getting a gpu for running some of these chatbots locally.

As a very occasional gamer who uses an iMac for work I thought about getting a gaming PC for like 6 years. Last fall it seemed that all the stars have aligned. The crypto winter and Ethereum switching to proof of stake meant that GPU prices fell to a reasonable level, I knew i would have a bit of a time to play some game during the holidays and as soon as Stable Diffusion was first posted on hacker news I knew that t…

When ever I feel like gaming I just subscribe to geforce now service. Around here it costs around ~$10 a month which I usually go go or ~$3 for a single day. And as the servers are located at a local isp no network latency or dropped packets.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#125

Post by F2hP18Foam »

swyx wrote: post #11
a5huynh wrote: post #10

Note that what they released are the delta weights from the og LLaMa model. To play around with it, you'll need to grab the original LLaMA 13B model and apply the changes. > We release Vicuna weights as delta weights to comply with the LLaMA model > license. You can add our delta to the original LLaMA weights to obtain > the Vicuna weights. Edit: took me a while to find it, here's a direct link to the delta weights:…

so an extra licensing issue to get around the original non commercial license... this is just a research curiosity is it not?

Seems that way, it would probably be a bad idea to use this for anything commercial at the very least.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#126

Post by pugio »

d4rkp4ttern wrote: post #122

I have an M1 MBP 64GB. Can I run it on my M1 Or do I need a GPU ?

I have it running, slowly, on the same machine. I would love for someone to get support running for MPS backend (The GPU) but it does run on the CPU.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#128

Post by aftbit »

capableweb wrote: post #64
m3kw9 wrote: post #57

Earlier quoted context omitted.

You can just say “no explain” and it won’t explain. Don’t waste tokens patronizing gpt lol

Doesn't always work. Sometimes you have to be more verbose or even repeat yourself in order to force the model to do something. This is one of those cases. Just adding "No explain" in the end of the prompt often doesn't stop it from adding an explanation anyways. > write a function in JavaScript that turns a JavaScript array into JS DOM elements, like what Hiccup does in Clojure. No explain Makes GPT-4 output text +…

Hah for GPT-3.5-Turbo I prompted:

    write a function in JavaScript that turns a JavaScript array into JS DOM elements, like what Hiccup does in Clojure. No explain
It replied:

    ation is necessary, just write the code.
    
    function arrayToDOM(arr) {
    (code follows)

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#129

Post by yreg »

xbmcuser wrote: post #124
yreg wrote: post #106

Earlier quoted context omitted.

As a very occasional gamer who uses an iMac for work I thought about getting a gaming PC for like 6 years. Last fall it seemed that all the stars have aligned. The crypto winter and Ethereum switching to proof of stake meant that GPU prices fell to a reasonable level, I knew i would have a bit of a time to play some game during the holidays and as soon as Stable Diffusion was first posted on hacker news I knew that t…

When ever I feel like gaming I just subscribe to geforce now service. Around here it costs around ~$10 a month which I usually go go or ~$3 for a single day. And as the servers are located at a local isp no network latency or dropped packets.

That would be more cost-efficient for me as well. But I somehow like owning the hardware instead of renting it. Oh well, at least I can now locally tinker with all the diffusion and LLM projects that are being released.

Re: State-of-the-art open-source chatbot, Vicuna-13B, just released model weights

#130

Post by m3kw9 »

capableweb wrote: post #64
m3kw9 wrote: post #57

Earlier quoted context omitted.

You can just say “no explain” and it won’t explain. Don’t waste tokens patronizing gpt lol

Doesn't always work. Sometimes you have to be more verbose or even repeat yourself in order to force the model to do something. This is one of those cases. Just adding "No explain" in the end of the prompt often doesn't stop it from adding an explanation anyways. > write a function in JavaScript that turns a JavaScript array into JS DOM elements, like what Hiccup does in Clojure. No explain Makes GPT-4 output text +…

True, it’s a fuzzy NN after all, but does more verbose really 100%? I’d still stick with it if it works 80% of the time. Or find a better short prompt like try “code only” or “just code”. Promoting can get really tiring.
Post reply on HN