Gemini "duck" demo was not done in realtime or with voice

Post by mvdtnz » Thu, Dec 07, 2023, 9:57 PM UTC

AndrewKemendo wrote: post #228

mvdtnz wrote: post #76

Earlier quoted context omitted.

Showing a product in its best light is one thing. Demonstrating a mode of operation that doesn't exist is entirely another. It would be like if a demo of your swipe keyboard included telepathic mind control for correcting errors.

I’m not sure I’d agree that what they showed will never be possible and in fact my whole point is that I think Google can most likely deliver on that in this specific case. Chalk it up to my experience in the space, but from what I can see it looks like something Google can actually execute on (unlike many areas where they fail on product regularly). I would agree completely that it’s not ready for consumers the way…

Post by zozbot234 » Thu, Dec 07, 2023, 9:58 PM UTC

dylan604 wrote: post #298

zozbot234 wrote: post #267

Earlier quoted context omitted.

How is that any different from what these AI chatbots are doing? They make stuff up that they predict will be rewarded highly by humans who look at it. This is exactly what leads to truisms like "rubber duckies are made of a material that floats over water" - which looks like it should be correct, even though it's wrong. It really is no different from Facebook memes that are devised to get a rise out of people and be…

Because we shouldn't be striving to make mediocrity. We should be striving to build better. Unless the devs of the bots are wanting to have a bot built on trying to deceive people, I just don't see the purpose of this. If we can "train" a bot and fine tune it, we should be fine tuning truth and telling it what absolutely is bullshit. To avoid the darker topics to keep the conversation on the rails, if there were a mi…

Post by zdrummond » Thu, Dec 07, 2023, 9:59 PM UTC

Post by modeless » Thu, Dec 07, 2023, 10:00 PM UTC

bitshiftfaced wrote: post #244

modeless wrote: post #229

Earlier quoted context omitted.

The problem is not with the model, but with its portrayal in the marketing materials. It's not even the fact that it lied, which is actually realistic. The problem is the lie was not called out as such. A better demo would have had the user note the issue and give the model the opportunity to correct itself.

But you yourself said that it was so convincing that the people doing the demo didn't recognize it as false, so how would they know to call it out as such? I suppose they could've deliberately found a hallucination and showcased it in the demo. In which case, pretty much every company's promo material is guilty of not showcasing negative aspects of their product. It's nothing new or unique to this case.

Post by modeless » Thu, Dec 07, 2023, 10:03 PM UTC

eurleif wrote: post #270

modeless wrote: post #94

That's not the only thing wrong. Gemini makes a false statement in the video, serving as a great demonstration of how these models still outright lie so frequently, so casually, and so convincingly that you won't notice, even if you have a whole team of researchers and video editors reviewing the output. It's the single biggest problem with LLMs and Gemini isn't solving it. You simply can't rely on them when correctn…

To be fair, one could describe the duck as being made of air and vinyl polymer, which in combination are less dense than water. That's not how humans would normally describe it, but that's kind of arbitrary; consider how aerogel is often described as being mostly made of air.