Gemini "duck" demo was not done in realtime or with voice • Hacker News Board

Re: Gemini "duck" demo was not done in realtime or with voice

#211

Post by lancesells » Thu, Dec 07, 2023, 9:01 PM UTC

epalm wrote: post #146

jmathai wrote: post #105

Earlier quoted context omitted.

This seems to be a common view among some folks. Personally, I'm impartial. Search or even asking other expert human beings are prone to provide incorrect results. I'm unsure where this expectation of 100% absolute correctness comes from. I'm sure there are use cases, but I assume it's the vast minority and most can tolerate larger than expected inaccuracies.

I know exactly where the expectation comes from. The whole world has demanded absolute precision from computers for decades. Of course, I agree that if we want computers to “think on their own“ or otherwise “be more human“ (whatever that means) we should expect a downgrade in correctness, because humans are wrong all the time.

Is it less reliable than an encyclopedia? It is less reliable than Wikipedia? Those aren't infallible but what's the expectation if it's wrong on something relatively simple?

With the rush of investment in dollars and to use these in places like healthcare, government, security, etc. there should be absolute precision.

Re: Gemini "duck" demo was not done in realtime or with voice

#212

Post by famouswaffles » Thu, Dec 07, 2023, 9:02 PM UTC

lxgr wrote: post #158

famouswaffles wrote: post #143

Earlier quoted context omitted.

Not to say this example was lying but they can lie just fine - https://arxiv.org/abs/2311.07590

They're lying in the same way that a sign that says "free cookies" is lying when there are actually no cookies. I think this is a different usage of the word, and we're pretty used to making the distinction, but it gets confusing with LLMs.

You are making an imaginary distinction that doesn't exist. It doesn't even make any sense in the context of the paper i linked.

The model consistently and purposefully withheld knowledge it was directly aware of. This is lying under any useful definition of the word. You're veering off into meaningless philosophy that has no bearing on outcomes and results.

Re: Gemini "duck" demo was not done in realtime or with voice

#213

Post by bitshiftfaced » Thu, Dec 07, 2023, 9:02 PM UTC

modeless wrote: post #94

That's not the only thing wrong. Gemini makes a false statement in the video, serving as a great demonstration of how these models still outright lie so frequently, so casually, and so convincingly that you won't notice, even if you have a whole team of researchers and video editors reviewing the output. It's the single biggest problem with LLMs and Gemini isn't solving it. You simply can't rely on them when correctn…

There's nothing wrong with what you're saying, but what do you suggest? Factuality is an area of active research, and Deepmind goes into some detail in their technical paper.

The models are too useful to say, "don't use them at all." Hopefully people will heed the warnings of how they can hallucinate, but further than that I'm not sure what more you can expect.

Re: Gemini "duck" demo was not done in realtime or with voice

#214

Post by swores » Thu, Dec 07, 2023, 9:03 PM UTC

heleninboodler wrote: post #117

kjkjadksj wrote: post #54

Earlier quoted context omitted.

Its honestly pretty mind boggling that we’d even use querty on a smartphone. The entire point of the layout is to keep your fingers on the home row. Meanwhile people text with a single or two thumbs 100% of the time.

The reason we use qwerty on a smartphone is extremely straightforward: people tend to know where to look for the keys already, so it's easy to adopt to even though it's not "efficient". We know it better than we know the positions of letters in the alphabet. You can easily see the difference if you're ever presented with an onscreen keyboard that's in alphabetical order instead of qwerty (TVs do this a lot, for some…

That's definitely a good reason why, but perhaps if iOS or Android were to research what the best layout is for typical touch screen typing and release that as a new default, people would find it quite quick to learn a second layout and soon get just the benefits?

After all, with TVs I've had the same experience as you with the annoying alphabetical keyboard, but we type into they maybe a couple of times a year, or maybe once in 5 years, whereas if we changed our phone keyboard layout we'd likely get used to it quite quickly.

Even if not going so far as to push it as a new default for all users (I'm willing to accept the possibility that I'm speaking for myself as the kind of geeky person who wouldn't mind the initial inconvenience of a new kb layout if it meant saving time in the long run, and that maybe a large majority of people would just hate it too much to be willing to give it a chance), they could at least figure out what the best layout is (maybe this has been studied and decided already, by somebody?) and offer that as an option for us geeks.

Re: Gemini "duck" demo was not done in realtime or with voice

#215

Post by skilled » Thu, Dec 07, 2023, 9:03 PM UTC

fake benchmarks, fake stitched together videos, disingenuous charts, no developer API on launch, announcements stuffed with marketing fluff.

As soon as I saw that opening paragraph from Sundar and how it was written I knew that Gemini is going to be a steaming pile of shit.

They should have watched the GPT-4 announcement from OpenAI again. That demo Greg Brockman did with converting a sketch on a piece of paper to a CodePen from a Discord channel, with all the error correcting and whatnot, is how you launch a product that's appealing to users.

TechCrunch, Twitter and some other sites (including HN i guess) are already piling on to this and by Monday things will go back to how they were and Google will have to go back to the drawing board to figure out another way to relaunch Gemini in the future.

Re: Gemini "duck" demo was not done in realtime or with voice

#216

Post by awongh » Thu, Dec 07, 2023, 9:03 PM UTC

ajkjk wrote: post #173

awongh wrote: post #161

Earlier quoted context omitted.

I’m not an expert but I suspect that this aspect of lack of correctness in these models might be fundamental to how they work. I suppose there’s two possible solutions: one is a new training or inference architecture that somehow understand “facts”. I’m not an expert so I’m not sure how that would work, but from what I understand about how a model generates text, “truth” can’t really be a element in the training or i…

> this aspect of lack of correctness in these models might be fundamental to how they work. Is there some sense in which this isn't obvious to the point of triviality? I keep getting confused because other people seem to keep being surprised that LLMs don't have correctness as a property. Even the most cursory understanding of what they're doing understands that it is, fundamentally, predicting words from other words…

Yeah. I think there's some ambiguity around the meaning of reasoning- because it is a kind of reasoning to say a Duck's material is less dense than water. In a way it's reasoned that out, and it might actually say something about the way a lot of human reasoning works.... (especially if you've ever listened to certain people talk out loud and say to yourself... huh?)

Re: Gemini "duck" demo was not done in realtime or with voice

#217

Post by twobitshifter » Thu, Dec 07, 2023, 9:04 PM UTC

modeless wrote: post #94

That's not the only thing wrong. Gemini makes a false statement in the video, serving as a great demonstration of how these models still outright lie so frequently, so casually, and so convincingly that you won't notice, even if you have a whole team of researchers and video editors reviewing the output. It's the single biggest problem with LLMs and Gemini isn't solving it. You simply can't rely on them when correctn…

I, a non-AGI, just ‘hallucinated’ yesterday. I hallucinated that my plan was to take all of Friday off and started wondering why I had scheduled morning meetings. I started canceling them in a rush. In fact, all week I had been planning to take a half day, but somehow my brain replaced the idea of a half day off with a full day off. You could have asked me and I would have been completely sure that I was taking all of friday off.

Re: Gemini "duck" demo was not done in realtime or with voice

#218

Post by hunter2_ » Thu, Dec 07, 2023, 9:04 PM UTC

onedognight wrote: post #167

modeless wrote: post #112

Earlier quoted context omitted.

Calling the air inside the duck (which is not sealed inside) part of its "material" would be misleading. That's not how most people would interpret the statement and I'm confident that's not the explanation for why the statement was made.

The air doesn’t matter. Even with a vacuum inside it would float. It’s the overall density of “the duck” that matters, not the density of the plastic.

A canoe floats, and that doesn't even command any thought regarding whether you can replace trapped air with a vacuum. If you had a giant cube half full of water, with a boat on the water, the boat would float regardless of whether the rest of the cube contained air or vacuum, and regardless of whether the boat traps said air (like a pontoon) or is totally vented (like a canoe). The overall density of the canoe is NOT influenced by its shape or any air, though. The canoe is strictly more dense than water (it will sink if it capsizes) yet in the correct orientation it floats.

What does matter, however, is the overall density of the space that was water and became displaced by the canoe. That space can be populated with dense water, or with a less dense canoe+air (or canoe+vacuum) combination. That's what a rubber duck also does: the duck+air (or duck+vacuum) combination is less dense than the displaced water.

Re: Gemini "duck" demo was not done in realtime or with voice

#219

Post by bongodongobob » Thu, Dec 07, 2023, 9:05 PM UTC

__s wrote: post #200

modeless wrote: post #94

That's not the only thing wrong. Gemini makes a false statement in the video, serving as a great demonstration of how these models still outright lie so frequently, so casually, and so convincingly that you won't notice, even if you have a whole team of researchers and video editors reviewing the output. It's the single biggest problem with LLMs and Gemini isn't solving it. You simply can't rely on them when correctn…

It also says the attribute of squeaking means it'll definitely float

That's actually pretty clever because if it squeaks, there is air inside. How many squeaking ducks have you come across that don't float?

Re: Gemini "duck" demo was not done in realtime or with voice

#220

Post by jerf » Thu, Dec 07, 2023, 9:07 PM UTC

bigtunacan wrote: post #91

jerf wrote: post #75

Earlier quoted context omitted.

"The entire point of the layout is to keep your fingers on the home row." No, that is how you're told to type . You have to be told to type that way precisely because QWERTY is not designed to keep your fingers on the home row. If you type in a layout that is designed to do that, you don't need to be told to keep your fingers on the home row, because you naturally will. Nobody really knows what the designers were thi…

Hold up young one. The reason for QWERTYs design has absolutely not been lost to history yet. The design was to spread out the hammers of the most frequently used letters to reduce the frequency of hammer jamming back when people actually used typewriters and not computers. The problem it attempted to improve upon, and which is was pretty effective at, is just a problem that no longer exists.

Also apocryphal: https://en.wikipedia.org/wiki/QWERTY#Contemporaneous_alterna...

And it does a bad job at it, which is further evidence that it was not the design consideration. People may not have been able to run a quick perl script over a few gigabytes of English text, but they would have gotten much closer if that was the desire. I don't believe that was their goal but they were just too stupid to get it even close to right.