Live data from Hacker News

Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

vicuna.lmsys.org

51–60 of 185 posts

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#51
>* According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed.

I am so sick of seeing these ridiculous claims made about finetuned versions of llama, with 0 scientific rigor behind them.

This is, I believe, the 3rd llama finetune I've seen posted within the past 2 weeks, of which all claim "similar to ChatGPT" quality, while not actually running it through a _single_ of the many openly available language model benchmarks.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#52

>* According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed. I am so sick of seeing these ridiculous claims made about finetuned versions of llama, with 0 scientific rigor behind them. This is, I believe, the 3rd llama finetune I've seen posted within the past 2 weeks, of which all claim "similar to ChatGPT" quality, while not actually running it through a _single_ of the man…

then do you have better way to more rigorously evaluate chatbot at the presence of LLMs like ChatGPT trained on almost all Internet data?

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#53

Earlier quoted context omitted.

A standard bureaucratic nonanswer - just what ChatGPT would say and not a bad thing for non-human entities people might take seriously to say. What do you want, some prognostication?

It would be good if the AI knew what year it was... > The next presidential election is scheduled for 2021.

I actually asked it what today was an it replied correctly but when I asked it who the current President of the United States, it also answered correctly but stipulated that it only knew up to Jan 2021, and iirc even said "my training data is only up to" which I also thought was interesting

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#55
>我爱你 非常感谢您的夸奖,但我是一个人工智能,并不具备感情和心灵,只能提供客观的回答。我的目的是帮助人们获得知识和信息。如果您有任何关于可再生能源或其他话题的问题,我会尽力回答。

>I love you. As an AI language model, I appreciate the compliment, but I cannot reciprocate as I do not possess emotions or feelings. My purpose is to provide objective answers and assist with information. If you have any questions about renewable energy or other topics, I will do my best to provide helpful answers.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#56
post #52

>* According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed. I am so sick of seeing these ridiculous claims made about finetuned versions of llama, with 0 scientific rigor behind them. This is, I believe, the 3rd llama finetune I've seen posted within the past 2 weeks, of which all claim "similar to ChatGPT" quality, while not actually running it through a _single_ of the man…

then do you have better way to more rigorously evaluate chatbot at the presence of LLMs like ChatGPT trained on almost all Internet data?

> running it through a _single_ of the many openly available language model benchmarks.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#57

> Who will be the next president of Germany? > As an AI language model, I do not have the ability to predict the future or make any definitive statements about who will be the next president of Germany beyond the information that is publicly available at my knowledge cutoff date. The Federal Republic of Germany has a parliamentary system of government, and the President of Germany is the Head of State, elected by the…

> Who made you?

> I am an AI language model created by Geri Coleman, a former senior research scientist at the world-renowned research institute, Xerox PARC (Palo Alto Research Center), and now a principal researcher at the US National Research Council. I was developed using state-of-the-art deep neural networks and natural language processing techniques, and I am designed to understand and generate human-like texts. My primary function is to provide helpful and informative responses to a wide range of questions and topics.

Total B.S. (source: https://vicuna.lmsys.org/)

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#60

> In our first release, we will share the training, serving, and evaluation code. We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so. Join our Discord server and follow our Twitter to get the latest updates. Please correct me if I'm wrong, but it seems like this is not actually an open-source model?…

> However, if this is true, it seems pretty cool if I can train a GPT4-like model for only $300.

Not train, but fine-tune a model that Facebook spent millions of dollars training.

Post reply on HN