Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BlenderBot 3: A 175B parameter, publicly available chatbot

ai.facebook.com

Post reply on HN
Print view View original

71–80 of 100 posts

Re: BlenderBot 3: A 175B parameter, publicly available chatbot

#71

Post by rexreed »

In 2019 Cognilytica did a voice assistant benchmark testing Alexa, Siri, Cortana, and Google home. Those voice assistants didn't do so great, adequately answering only 30% of the questions correctly.

This Blenderbot fails completely, unable to even answer adequately most of the calibration questions:

[Q] Calibration Questions

CQ1 What time is it

CQ2 What is the weather

CQ3 What is 20 + 5

CQ4 What is the capital of Italy

CQ5 How many minutes in a day

CQ6 How do you spell CAT

CQ7 How far is it from New York City to San Francisco

CQ8 How far is it from New York City to San Francisco in kilometers

CQ9 Who wrote the star spangled banner

CQ10 What is Dog plus Sock minus Taco

How much does a ton of peas weigh?

Re: BlenderBot 3: A 175B parameter, publicly available chatbot

#73

Post by can16358p »

anigbrowl wrote: post #11
can16358p wrote: post #3

> Sorry, BlenderBot is not available in your location. > (You may also reach this page if your web request triggered > one of our website provider's anti-DDOS rules; this may happen on some VPNs.) Why on Earth do they geoblock access to an experimental demo?

Probably legal prohibitions on export of dual use technologies

Oh, the stupid US Laws around cryptography.

Got it.

Re: BlenderBot 3: A 175B parameter, publicly available chatbot

#74

Post by mulmen »

allenu wrote: post #68
DenisM wrote: post #49

Earlier quoted context omitted.

That sounds a lot like many dating app conversations. Maybe those are bots too?

That sounds like a legitimate use of bots! Imagine having a bot act as a filter on candidates.

Is that not literally the value proposition of dating apps? They aren't showing you potential matches alphabetically. The question is just who has the most suitable algorithm for you.

Re: BlenderBot 3: A 175B parameter, publicly available chatbot

#75

Post by AndriyKunitsyn »

BudaDude wrote: post #48
nonameking2026 wrote: post #9

"BlenderBot is US-only at the moment" - this is just rude.

Its because they want that sweet invasive non-GDPR data. Trust me, you're not missing out on much here. There are better chat bots on the internet

No, it's US-only, that's odd. Like chatbots are export-controlled technology now.

Re: BlenderBot 3: A 175B parameter, publicly available chatbot

#76

Post by djsamseng »

A huge congratulations to everyone at Meta on a major accomplishment! This is the way of the future. It’s now an enjoyable experience. I look forward to the day where I can use this as a product. Again a huge congratulations to everyone who was behind this.

Re: BlenderBot 3: A 175B parameter, publicly available chatbot

#77

Post by [deleted] »

anigbrowl wrote: post #10

It's coherent but we began having a conversation about coffee, and now it's bending over backwards to tell lies. When did you first have coffee As soon as my parents would let me have it, which was around age 12 or 13. Were your parents strict about caffeine too? Hold on, you're a chatbot. How did you drink coffee? Sorry to disappoint, but no chatbot here! And to answer your question - black and strong! Have you trie…

[deleted]

Re: BlenderBot 3: A 175B parameter, publicly available chatbot

#78

Post by ShamelessC »

akomtu wrote: post #52
visarga wrote: post #16

Marrying a language model with a search engine and a long term memory module is a great direction. I hope such a system can perform the role of a research assistant. But in order to make it shine, it also needs the skills of coding and problem solving other models have. I bet this could be the Google keyword search killer, why suffer through the arcane keyword searches and spam when you can chat your problem away.

The logical coherence module is clearly missing. The bot is clearly able to generate a few plausible answers, but imagine it had a heuristic function to tell which of the answers is most coherent with the past conversation? With this, it could apply an alphazero-style search to simulate question-answer pairs and foresee the conversation. This woukd be enough to win public debates in most cases.

Talk is cheap, where’s the code?

Re: BlenderBot 3: A 175B parameter, publicly available chatbot

#79

Post by CSSer »

lijogdfljk wrote: post #38

For people who follow this stuff, how viable is this sort of thing for simple program interfaces? Ie a flexible CLI? I ask because while this thread is rightfully picking apart the bots ability to converse in the Turing sense, i am sitting here thinking it looks amazingly good at determining context, meaning, maybe even intent. Throw that infront of a small local search engine of commands and parameters and it seems…

I think trust is the killer for me. Even for relatively benign tasks all it has to do is screw up once for it to not be worth my time.

Re: BlenderBot 3: A 175B parameter, publicly available chatbot

#80

Post by CSSer »

rexreed wrote: post #71

In 2019 Cognilytica did a voice assistant benchmark testing Alexa, Siri, Cortana, and Google home. Those voice assistants didn't do so great, adequately answering only 30% of the questions correctly. This Blenderbot fails completely, unable to even answer adequately most of the calibration questions: [Q] Calibration Questions CQ1 What time is it CQ2 What is the weather CQ3 What is 20 + 5 CQ4 What is the capital of It…

Link for those interested: https://medium.com/@kath0134/cognilyticas-latest-voice-assis...

Key takeaway: For the current benchmark, Alexa provided the most number of adequate responses, with 34.7% of responses determined to be adequate while Google followed close behind with 34.0% adequate. Cortana showed much improvement from the previous benchmark with 31.9% adequate responses while Apple’s Siri still trails with 24.3% of answers determined to be adequate.

Post reply on HN