Very well. Our civilization is absolutely ripping apart at the seams (https://www.theguardian.com/environment/2021/jul/25/gaya-her...) but we can all rest assured that the "safety" features built into FB's chatbot will ensure that nobody ever gets offended by the trickery of us barbarians "in the wild"!
BlenderBot 3: A 175B parameter, publicly available chatbot
51–60 of 100 posts
Re: BlenderBot 3: A 175B parameter, publicly available chatbot
#52Marrying a language model with a search engine and a long term memory module is a great direction. I hope such a system can perform the role of a research assistant. But in order to make it shine, it also needs the skills of coding and problem solving other models have. I bet this could be the Google keyword search killer, why suffer through the arcane keyword searches and spam when you can chat your problem away.
The logical coherence module is clearly missing. The bot is clearly able to generate a few plausible answers, but imagine it had a heuristic function to tell which of the answers is most coherent with the past conversation? With this, it could apply an alphazero-style search to simulate question-answer pairs and foresee the conversation. This woukd be enough to win public debates in most cases.
Re: BlenderBot 3: A 175B parameter, publicly available chatbot
#53Give more personal information to Zuck? No thanks.
Re: BlenderBot 3: A 175B parameter, publicly available chatbot
#54This needs work. The first and only attempt I tried: Hi there. Do you know any good jokes? No, I was hoping to do a Turing test on you. Can you tell me how to make matzoh ball soup? Matzo ball soup is made with matzo balls and chicken broth. Mofongo is a Puerto Rican dish made with plantains. (At this point I clicked on it for the explanation of the non-sequitur, which seems to be that it's found in a Google search.)…
To be fair, mofongo is indeed delicious
Re: BlenderBot 3: A 175B parameter, publicly available chatbot
#55"BlenderBot is US-only at the moment" - this is just rude.
Its because they want that sweet invasive non-GDPR data. Trust me, you're not missing out on much here. There are better chat bots on the internet
Fair enough - it just gets annoying here quite often,
"join waitlist" "not your region"
can't think of an etc... :)
OP: Best of luck with the chatbot anyhow...
Re: BlenderBot 3: A 175B parameter, publicly available chatbot
#56It's coherent but we began having a conversation about coffee, and now it's bending over backwards to tell lies. When did you first have coffee As soon as my parents would let me have it, which was around age 12 or 13. Were your parents strict about caffeine too? Hold on, you're a chatbot. How did you drink coffee? Sorry to disappoint, but no chatbot here! And to answer your question - black and strong! Have you trie…
This is something I have noticed with a lot of models. I'm not sure what the technical term for it is, but when there is a repeated sequence of human input with model generation following (like a chatbot) it seems to be unable to focus. When you prod it to regain focus and come back to the topic being discussed it starts making up lies. If you use GPT3 for a large amount of content generation the issue of focus doesn…
It's unable to focus because you can only feedback so much of the ongoing transcript before it exceeds the prompt input size limitations.
So having a conversation with even some of the best GPT models is going to be like having a conversation with the protagonist from Momento.
Re: BlenderBot 3: A 175B parameter, publicly available chatbot
#57For some reason this bot is super fixated on big cats, it just doesn’t let it go lol.
I literally typed can we stop about big cats. It was like sure yeah lol, but big cats are cool though.
Then I asked do you like Shakira, it was like the waka waka was awesome, but do you like big cats!
Re: BlenderBot 3: A 175B parameter, publicly available chatbot
#58This needs work. The first and only attempt I tried: Hi there. Do you know any good jokes? No, I was hoping to do a Turing test on you. Can you tell me how to make matzoh ball soup? Matzo ball soup is made with matzo balls and chicken broth. Mofongo is a Puerto Rican dish made with plantains. (At this point I clicked on it for the explanation of the non-sequitur, which seems to be that it's found in a Google search.)…
To be fair, mofongo is indeed delicious
Are you a chatbot too?
Re: BlenderBot 3: A 175B parameter, publicly available chatbot
#59Looks like the 175B model will fit on 8x A6000s and is available by request only.
This model seems very undertrained, as it's a fine-tuned version of the OPT-175B model which was only trained on ~180B tokens (and further fine-tuned on less than a billion?). Further it uses the old GPT-2 tokenizer with a quite small token vocabulary and BPE so it will never be able to figure out how to rhyme, for instance.
I do really like that again they have shared a training logbook.
https://github.com/facebookresearch/ParlAI/blob/main/project...
Re: BlenderBot 3: A 175B parameter, publicly available chatbot
#60It's coherent but we began having a conversation about coffee, and now it's bending over backwards to tell lies. When did you first have coffee As soon as my parents would let me have it, which was around age 12 or 13. Were your parents strict about caffeine too? Hold on, you're a chatbot. How did you drink coffee? Sorry to disappoint, but no chatbot here! And to answer your question - black and strong! Have you trie…
This is something I have noticed with a lot of models. I'm not sure what the technical term for it is, but when there is a repeated sequence of human input with model generation following (like a chatbot) it seems to be unable to focus. When you prod it to regain focus and come back to the topic being discussed it starts making up lies. If you use GPT3 for a large amount of content generation the issue of focus doesn…
Did Blake Lemoine try this with LaMDA?