Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

vicuna.lmsys.org

Post reply on HN
Print view View original

101–110 of 185 posts

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#101

Post by gpm »

pazimzadeh wrote: post #11

> Can you name all of the soccer players whose names contain a Z > Zlatan Ibrahimovic Zinedine Zidane Zoran Banović Zlatko Zahovič Zoran Matić Zayed Khoury Zé Mario Zé Douala Zé Elias Zé Luís Zé Manuel Zé Roberto Zé Xavier Zé Abel Zé Eduardo Zé Tomás Zé Luís Zé Henrique Zé Jorge Zé Breno Zé Manuel Zé Luís Zé Sérgio Zé Maria Zé Roberto Zé Ricardo Zé Álvaro Zé Luís Zé Carlos Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé L…

Considering words are mostly tokenized I'm not really surprised that questions about spelling don't work so well.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#102

Post by Matumio »

pazimzadeh wrote: post #11

> Can you name all of the soccer players whose names contain a Z > Zlatan Ibrahimovic Zinedine Zidane Zoran Banović Zlatko Zahovič Zoran Matić Zayed Khoury Zé Mario Zé Douala Zé Elias Zé Luís Zé Manuel Zé Roberto Zé Xavier Zé Abel Zé Eduardo Zé Tomás Zé Luís Zé Henrique Zé Jorge Zé Breno Zé Manuel Zé Luís Zé Sérgio Zé Maria Zé Roberto Zé Ricardo Zé Álvaro Zé Luís Zé Carlos Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé L…

Asking a token-based LM to generate only tokens that contain the letter "Z" is... well.

Like asking someone who prepared a speech to smoothly skip all words with "ei" in them. The LM would basically need to have memorized the set of letters that each of its token stands for, from the training data. I'm surprised it works at all.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#103

Post by visarga »

jerojero wrote: post #47

The work being done with Llama and Alpaca and so on is what I've found to be most interesting in the past few weeks. The fact that we are capable of producing such high quality conversational models that we can actually get to freely experiment with is really encouraging. I'm particularly excited about applications such as viper[1] and other forms of plugins as well as model ensembles. [1] https://viper.cs.columbia.e…

Turns out LLMs are not so inaccessible for individual users.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#104

Post by visarga »

nenkoru wrote: post #33

Q: We need you to draw seven red lines. All of them strictly perpendicular; some with green ink and some with transparent. Can you do that? A: I'm sorry, but I am a text-based AI model and I don't have the ability to physically draw lines or use different colors of ink. However, I can help you visualize this concept by describing the red lines and the different ink colors. A red line with green ink: Imagine a traditi…

Well, technically you could draw seven perpendicular lines in 7D. It's just the coordinate system.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#105

Post by curiousmindz »

pazimzadeh wrote: post #11

> Can you name all of the soccer players whose names contain a Z > Zlatan Ibrahimovic Zinedine Zidane Zoran Banović Zlatko Zahovič Zoran Matić Zayed Khoury Zé Mario Zé Douala Zé Elias Zé Luís Zé Manuel Zé Roberto Zé Xavier Zé Abel Zé Eduardo Zé Tomás Zé Luís Zé Henrique Zé Jorge Zé Breno Zé Manuel Zé Luís Zé Sérgio Zé Maria Zé Roberto Zé Ricardo Zé Álvaro Zé Luís Zé Carlos Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé L…

I just tried with Bing AI (in "precise" mode), it ran a search, then answered:

> There are many soccer players whose names contain a Z. Some of them include Zlatan Ibrahimović, Zinedine Zidane, Zico, Zvonimir Boban, and Zé Roberto ¹. Would you like to know more about any specific player?

Source: Conversation with Bing, 3/31/2023 ...

I also tried: "Can you name 14 soccer players whose names contain a Z?" (At 15+, it answers "I’m sorry, but I don’t have 15 names of soccer players whose names contain a Z. However, I can provide you with a few more names")

> Sure! Here are 14 soccer players whose names contain a Z: Zlatan Ibrahimović, Zinedine Zidane, Zico, Zvonimir Boban, Zé Roberto, Zbigniew Boniek, Zakaria Aboukhlal, Zlatko Zahovič, Zubayr Amiri, Zico, Zé Luís, Zećira Mušović, Zsolt Lőw and Zah Rahan Krangar. Is there anything else you would like to know?

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#106

Post by visarga »

pazimzadeh wrote: post #11

> Can you name all of the soccer players whose names contain a Z > Zlatan Ibrahimovic Zinedine Zidane Zoran Banović Zlatko Zahovič Zoran Matić Zayed Khoury Zé Mario Zé Douala Zé Elias Zé Luís Zé Manuel Zé Roberto Zé Xavier Zé Abel Zé Eduardo Zé Tomás Zé Luís Zé Henrique Zé Jorge Zé Breno Zé Manuel Zé Luís Zé Sérgio Zé Maria Zé Roberto Zé Ricardo Zé Álvaro Zé Luís Zé Carlos Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé L…

Not even GPT-4 can do it. It starts with two correct names, then just adds a bunch of names with no letter Z in them. When prompted to fix, it makes the same error.

My theory is that once it starts making a list, it falls into a different mode and forgets the filtering rule. Like mode collapse in GANs.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#107

Post by visarga »

brianjking wrote: post #4

This is great, but similar to GPT4All, it will likely be deemed unusable for any commercial or otherwise "legitimate" use cases since it's trained on OpenAI completions from sharegpt.com. https://github.com/nomic-ai/gpt4all

I honestly think it's the better way to deal with this problem - nothing the model generates should be copyrightable. You can use model outputs for anything unless the model replicates training data verbatim. This leaves a path open for AI skills to trickle down to open source models. It's a pity we can't copyright model outputs or the models themselves (also a result of a mechanistic process), but better in the long run.

We should not protect ideas from replication, only the expression should be copyrightable. Using data from another model extracts the ideas without the expression of the original training set, exactly following the idea/expression rule.

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#108

Post by number6 »

techdragon wrote: post #93
MMMercy2 wrote: post #69

Earlier quoted context omitted.

I am a Vicuna developer. We plan to release the weights once we have addressed all concerns and have a low-resource version of the inference code ready. We released the demo first to get some early feedback on the model.

We hear a lot about "concerns" and many of us don't share the same ones... It would be good for clarity to know what are the concerns you feel are important enough to hold back releasing the weights?

Financial and Political would be my guess. But maybe I just want to tease out an answer...

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#109

Post by pazimzadeh »

Matumio wrote: post #102
pazimzadeh wrote: post #11

> Can you name all of the soccer players whose names contain a Z > Zlatan Ibrahimovic Zinedine Zidane Zoran Banović Zlatko Zahovič Zoran Matić Zayed Khoury Zé Mario Zé Douala Zé Elias Zé Luís Zé Manuel Zé Roberto Zé Xavier Zé Abel Zé Eduardo Zé Tomás Zé Luís Zé Henrique Zé Jorge Zé Breno Zé Manuel Zé Luís Zé Sérgio Zé Maria Zé Roberto Zé Ricardo Zé Álvaro Zé Luís Zé Carlos Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé L…

Asking a token-based LM to generate only tokens that contain the letter "Z" is... well. Like asking someone who prepared a speech to smoothly skip all words with "ei" in them. The LM would basically need to have memorized the set of letters that each of its token stands for, from the training data. I'm surprised it works at all.

It would be nice if it would tell you when you are asking it something that it is not designed to answer

Re: Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

#110

Post by Technetium »

mdaniel wrote: post #53
twblalock wrote: post #17

Earlier quoted context omitted.

It would be good if the AI knew what year it was... > The next presidential election is scheduled for 2021.

I actually asked it what today was an it replied correctly but when I asked it who the current President of the United States, it also answered correctly but stipulated that it only knew up to Jan 2021, and iirc even said "my training data is only up to" which I also thought was interesting

The response content there almost certainly comes from the training data from ShareGPT.
Post reply on HN