Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini "duck" demo was not done in realtime or with voice

twitter.com

Post reply on HN
Print view View original

421–430 of 683 posts

Re: Gemini "duck" demo was not done in realtime or with voice

#421

Post by TerrifiedMouse »

adriand wrote: post #413
calf wrote: post #330

Earlier quoted context omitted.

Ever since the "stochastic parrots" and "super-autocomplete" criticisms of LLMs, the question is whether hallucinations are solvable in principle at all. And if hallucinations are solvable, it would of such basic and fundamental scientific importance that I think would be another mini-breakthrough in AI.

An interesting perspective on this I’ve heard discussed is whether hallucinations ought to be solved at all, or whether they are core to the way human intelligence works as well, in the sense that that is what is needed to produce narratives. I believe it is Hinton that prefers “confabulation” to “hallucination” because it’s more accurate. The example in the discussion about hallucination/confabulation was that of so…

> Without the ability to confabulate, he would have been unable to tell his story.

You can tell a story without making up fiction. Just say you don’t know when you don’t know.

Inaccurate information is worse than no information.

Re: Gemini "duck" demo was not done in realtime or with voice

#422

Post by dmix »

impulser_ wrote: post #416
DonnyV wrote: post #259

This is so crazy. Google invented transformers which is the bases for all these models. How do they keep fumbling like this over and over. Google Docs created in 2006! Microsoft is eating their lunch. Google creates the ability to change VM's in place and makes a fully automated datacenter. Amazon and Microsoft are killing them in the cloud. Google has been working on self driving longer than anyone. Tesla is catchin…

You bring up fumbles, but they still have the more products with more than a billion users than any company in the world. This is what Google has always cared about. Bring application to the billions of users. People are forgetting Google is the most profitable AI company in the world right now. All of their products use ML and AI. So who is losing? The goal of Gemini isn't to build a chatbot like ChatGPT despite Goo…

This is like critiquing Disney for putting out garbage and then defending them because dummies keep giving them money regardless of quality. Having standards and expectations of greatness is a good thing and the last thing you want is for mediocrity to become acceptable in society.

Re: Gemini "duck" demo was not done in realtime or with voice

#423

Post by TerrifiedMouse »

plaidfuji wrote: post #337
davesque wrote: post #287

A big red flag for me was that Sundar was prompting the model to report lots of facts that can be either true or false. We all saw the benchmark figures that they published and the results mostly showed marginal improvements. In other words, the issue of hallucination has not been solved. But the demo seemed to imply that it had. My conclusion was that they had mostly cherry picked instances in which the model happen…

These LLMs do not have a concept of factual correctness and are not trained/optimized as such. I find it laughable that people expect these things to act like quiz bots - this misunderstands the nature of a generative LLM entirely. It simply spits out whatever output sequence it feels is most likely to occur after your input sequence. How it defines “most likely” is the subject of much research, but to optimize for f…

> In the end the quality of the information it will get back to you is no better than the quality of a thorough google search.. it will just get you a more concise and well-formatted answer faster.

I would say it’s worse than Google search. Google tells you when it can’t find what you are looking for. LLMs “guess” a bullshit answer.

Re: Gemini "duck" demo was not done in realtime or with voice

#424

Post by jonplackett »

pphysch wrote: post #418
jonplackett wrote: post #402

OK I get that everyone’s hype sensitive and I absolutely remain to be convinced on Gemini’s actual ability BUT The fact this wasn’t realtime or with voice is not the issue. Voice to text could absolutely capture this conversation easily. And from what I’ve seen Gemini seems quicker than GPT4 Being able to respond quicker and via voice chat is not actually a big deal. The underlying performance of the model is what we…

Exactly. Corporate tech demo exaggerates actual capabilities and smoothes over rough edges? Impossible, this is unprecedented!! The Apple vs Google brand war is so tiresome. Let's focus on the tech.

To be clear - I’m not saying this makes Gemini good. Just that it isn’t bad for these reasons!

Re: Gemini "duck" demo was not done in realtime or with voice

#425

Post by dvsfish »

I did this at university. It was our first comp sci class ever, we were given raspberry pi's. We had no coding experience or guidance, and were asked to create "something". All we had to work with was information on how to communicate with the pi using putty. Oddly, this assignment didn't require us to submit code, but simply demonstrate it working.

My group (3 of us) bought a moisture sensor to plug into the pi, and had the idea to make a "flood detection system" that would be housed under a bridge, and would send an email to relevant people when the bridge home from work is about to flood.

So for our demonstration, we had a guy in the back of the class with gmail open ready to send an email saying some variation of "flood warning". Our script was literally just printing lines with wait statements in between. Running the script, it prints to the screen "awaiting moisture", and after 3 seconds it will print "moisture detected". In that 3 seconds I dip the sensor into the glass of water. Then the script would wait a few more seconds before printing "sending email to xxx@yyy.com". We then opened up our email, our mate at the back of the room hit send, and an email appeared saying flood warning, and we would get full marks.

Re: Gemini "duck" demo was not done in realtime or with voice

#426

Post by windowshopping »

rkeene2 wrote: post #417
freejazz wrote: post #207

Earlier quoted context omitted.

Maybe not if its the case that your use of the broken tool would result in the eventual undoing of your work. Like, lets say your staple gun is defective and doesn't shoot the staples deep enough, but it still shoots. You can keep using the gun, but it's not going to actually do its job. It seems useful and functional, but it isn't and its liable to create a much bigger mess.

So to continue the analogy, if the staple gun is broken and it requires you to do more than a working (but non-existent) staple gun BUT less work than doing the affixment without the broken staple gun, you would or would not use it ?

But nobody said they wouldn't use it. You said that. You came up with this idea and then demanded other people defend it.

I don't know why "critiquing the tool" is being equated to "refusing to use the tool."

I don't like calling something a strawman, because I think it's an overused argument, but...I mean...

Re: Gemini "duck" demo was not done in realtime or with voice

#427

Post by nmfisher »

jmathai wrote: post #166
stefan_ wrote: post #148

Earlier quoted context omitted.

You think you ask your legal assistant to find some precedents related to your current case and they will come back with an A4 page full of made up cases that sound vaguely related and convincing but are not real ? I don't think you understand the failure case at all.

That example seems a bit hyperbolic. Do you think lawyers who leverage ChatGPT will take the made up cases and present them to a judge without doing some additional research? What I'm saying is that the tolerance for mistakes is strongly correlated to the value ChatGPT creates. I think both will need to be improved but there's probably more opportunity in creating higher value. I don't have a horse in the race.

> Do you think lawyers who leverage ChatGPT will take the made up cases and present them to a judge without doing some additional research?

Oh dear.

Re: Gemini "duck" demo was not done in realtime or with voice

#428

Post by igor47 »

hot_gril wrote: post #301
DonnyV wrote: post #259

This is so crazy. Google invented transformers which is the bases for all these models. How do they keep fumbling like this over and over. Google Docs created in 2006! Microsoft is eating their lunch. Google creates the ability to change VM's in place and makes a fully automated datacenter. Amazon and Microsoft are killing them in the cloud. Google has been working on self driving longer than anyone. Tesla is catchin…

Engineer-driven company. Not enough top-down direction on the products. Too much self-perceived moral high ground. But lately they've been changing this.

My engineer friend who work at Google would strongly disagree with this assertion. I keep hearing about all sorts of hijinks initiated by senior PMs and managers trying to build their fiefdoms.

Re: Gemini "duck" demo was not done in realtime or with voice

#429

Post by Uptrenda »

DonnyV wrote: post #259

This is so crazy. Google invented transformers which is the bases for all these models. How do they keep fumbling like this over and over. Google Docs created in 2006! Microsoft is eating their lunch. Google creates the ability to change VM's in place and makes a fully automated datacenter. Amazon and Microsoft are killing them in the cloud. Google has been working on self driving longer than anyone. Tesla is catchin…

Big companies are where innovation goes to die.

Re: Gemini "duck" demo was not done in realtime or with voice

#430

Post by atleastoptimal »

DonnyV wrote: post #259

This is so crazy. Google invented transformers which is the bases for all these models. How do they keep fumbling like this over and over. Google Docs created in 2006! Microsoft is eating their lunch. Google creates the ability to change VM's in place and makes a fully automated datacenter. Amazon and Microsoft are killing them in the cloud. Google has been working on self driving longer than anyone. Tesla is catchin…

It's reassuring that the biggest tech company doesn't automatically make the best tech. If it were guaranteed that Google's resources would automatically trump any startup in the AI field, then it would likely predict a guaranteed dominance of incumbents and consolidation of power in the AI space.
Post reply on HN