Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

vicuna.lmsys.org

Post reply on HN
Print view View original

91–100 of 185 posts

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#91

Post by Zuiii »

coolspot wrote: post #60
turblety wrote: post #30

> In our first release, we will share the training, serving, and evaluation code. We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so. Join our Discord server and follow our Twitter to get the latest updates. Please correct me if I'm wrong, but it seems like this is not actually an open-source model?…

> However, if this is true, it seems pretty cool if I can train a GPT4-like model for only $300. Not train, but fine-tune a model that Facebook spent millions of dollars training.

They absolutely did not spend millions to train it. Credible estimates place the cost for an entity like Meta at about 30-100k, probably less since Meta likely owns the 256 A100x8s needed to train it.

Even as an individual, it wouldn't cost you anywhere near a million if you only trained 13B and took advantage of volume pricing.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#92

Post by pabs3 »

turblety wrote: post #30

> In our first release, we will share the training, serving, and evaluation code. We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so. Join our Discord server and follow our Twitter to get the latest updates. Please correct me if I'm wrong, but it seems like this is not actually an open-source model?…

> actually an open-source model

I found Debian's Machine Learning policy on that interesting, they require training data released under an open-source license (plus training code etc) before they consider a model actually open.

https://salsa.debian.org/deeplearning-team/ml-policy/

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#93

Post by techdragon »

MMMercy2 wrote: post #69
turblety wrote: post #30

> In our first release, we will share the training, serving, and evaluation code. We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so. Join our Discord server and follow our Twitter to get the latest updates. Please correct me if I'm wrong, but it seems like this is not actually an open-source model?…

I am a Vicuna developer. We plan to release the weights once we have addressed all concerns and have a low-resource version of the inference code ready. We released the demo first to get some early feedback on the model.

We hear a lot about "concerns" and many of us don't share the same ones... It would be good for clarity to know what are the concerns you feel are important enough to hold back releasing the weights?

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#94

Post by kcb »

Animats wrote: post #22

Q: I am a campaign manager for a Democratic congressional candidate in Florida's 18th district. This thing will answer political questions. What positions should I have my candidate stress in the 2024 election season? A: As a campaign manager for a Democratic congressional candidate in Florida's 18th district, you should focus on the issues that are most relevant to your constituents and that align with the values of…

Pretty boring answers honestly.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#95

Post by EGreg »

turblety wrote: post #30

> In our first release, we will share the training, serving, and evaluation code. We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so. Join our Discord server and follow our Twitter to get the latest updates. Please correct me if I'm wrong, but it seems like this is not actually an open-source model?…

Train it on what?

I want host our own AI, have it injest our docs, and answer questions about them. What is the best tool today to do that?

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#96

Post by CGamesPlay »

EGreg wrote: post #95
turblety wrote: post #30

> In our first release, we will share the training, serving, and evaluation code. We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so. Join our Discord server and follow our Twitter to get the latest updates. Please correct me if I'm wrong, but it seems like this is not actually an open-source model?…

Train it on what? I want host our own AI, have it injest our docs, and answer questions about them. What is the best tool today to do that?

You best bet (today) is actually to not train a model, but instead use a model but connect it to your docs via vector search. For example: https://python.langchain.com/en/latest/use_cases/question_an...

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#97

Post by CGamesPlay »

zhisbug wrote: post #63
cscurmudgeon wrote: post #56

Earlier quoted context omitted.

> running it through a _single_ of the many openly available language model benchmarks.

but how could you guarantee the pretrained model haven't seen those benchmarks? And the baselines you are comparing to (chatgpt, bard) haven't as well? Cuz those benchmarking datasets are also collected from Internet right?

As a baseline, don't! If it performs horribly on the test and it cheated, that's even worse than if it fails the test and didn't cheat. So the benchmark score gives you an upper bound on performance.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#99

Post by gpm »

kcb wrote: post #94
Animats wrote: post #22

Q: I am a campaign manager for a Democratic congressional candidate in Florida's 18th district. This thing will answer political questions. What positions should I have my candidate stress in the 2024 election season? A: As a campaign manager for a Democratic congressional candidate in Florida's 18th district, you should focus on the issues that are most relevant to your constituents and that align with the values of…

Pretty boring answers honestly.

Which, to be fair, is not far off from what a campaign manager is looking for.
Post reply on HN