Gemini "duck" demo was not done in realtime or with voice • Hacker News Board

Re: Gemini "duck" demo was not done in realtime or with voice

#251

Post by mellinoe » Thu, Dec 07, 2023, 9:25 PM UTC

swores wrote: post #214

heleninboodler wrote: post #117

Earlier quoted context omitted.

The reason we use qwerty on a smartphone is extremely straightforward: people tend to know where to look for the keys already, so it's easy to adopt to even though it's not "efficient". We know it better than we know the positions of letters in the alphabet. You can easily see the difference if you're ever presented with an onscreen keyboard that's in alphabetical order instead of qwerty (TVs do this a lot, for some…

That's definitely a good reason why, but perhaps if iOS or Android were to research what the best layout is for typical touch screen typing and release that as a new default, people would find it quite quick to learn a second layout and soon get just the benefits? After all, with TVs I've had the same experience as you with the annoying alphabetical keyboard, but we type into they maybe a couple of times a year, or m…

Even most technically-minded people still use QWERTY on full-size computer keyboards despite it being a terrible layout for a number of reasons. I really doubt a new, nonstandard keyboard would get much if any traction on phones.

Re: Gemini "duck" demo was not done in realtime or with voice

#252

Post by dr_dshiv » Thu, Dec 07, 2023, 9:26 PM UTC

dylan604 wrote: post #179

bostonpete wrote: post #162

Earlier quoted context omitted.

Speech to text is often wrong too. So is autocorrect. And object detection. Computers don't have to be 100% correct in order to be useful, as long as we don't put too much faith in them.

Your caveat is not the norm though, as everyone is putting a lot of faith in them. So, that's part of the problem. I've talked with people that aren't developers, but they are otherwise smart individuals that have absolutely not considered that the info is not correct. The readers here are a bit too close to the subject, and sometimes I think it is easy to forget that the vast majority of the population do not truly…

Nah, I don’t think anything has the potential to build critical thinking like LLMs en masse. I only worry that they will get better. It’s when they are 99.9% correct we should worry.

Re: Gemini "duck" demo was not done in realtime or with voice

#253

Post by umvi » Thu, Dec 07, 2023, 9:26 PM UTC

bitvoid wrote: post #188

kamikaz1k wrote: post #163

Earlier quoted context omitted.

Okay, but search is done on a computer, and like the person you’re replying to said, we accept close enough. I don’t necessarily disagree with your interpretation, but there’s a revealed preference thing going on. The number of non-tech ppl I’ve heard directly reference ChatGPT now is absolutely shocking.

> The number of non-tech ppl I've heard directly reference ChatGPT now is absolutely shocking. The problem is that a lot of those people will take ChatGPT output at face value. They are wholly unaware that of its inaccuracies or that it hallucinates. I've seen it too many times in the relatively short amount of time that ChatGPT has been around.

So you're saying we need a Ministry of Truth to protect people from themselves? This is the same argument used to suppress "harmful" speech on any medium.

Re: Gemini "duck" demo was not done in realtime or with voice

#254

Post by janalsncm » Thu, Dec 07, 2023, 9:27 PM UTC

ajkjk wrote: post #173

awongh wrote: post #161

Earlier quoted context omitted.

I’m not an expert but I suspect that this aspect of lack of correctness in these models might be fundamental to how they work. I suppose there’s two possible solutions: one is a new training or inference architecture that somehow understand “facts”. I’m not an expert so I’m not sure how that would work, but from what I understand about how a model generates text, “truth” can’t really be a element in the training or i…

> this aspect of lack of correctness in these models might be fundamental to how they work. Is there some sense in which this isn't obvious to the point of triviality? I keep getting confused because other people seem to keep being surprised that LLMs don't have correctness as a property. Even the most cursory understanding of what they're doing understands that it is, fundamentally, predicting words from other words…

Just to play devil’s advocate: we can train neural networks to model some functions exactly, given sufficient parameters. For example simple functions like ax^2 + bx + c.

The issue is that “correctness” isn’t a differentiable concept. So there’s no gradient to descend. In general, there’s no way to say that a sentence is more or less correct. Some things are just wrong. If I say that human blood is orange that’s not more incorrect than saying it’s purple.

Re: Gemini "duck" demo was not done in realtime or with voice

#255

Post by davesque » Thu, Dec 07, 2023, 9:27 PM UTC

bongodongobob wrote: post #219

__s wrote: post #200

Earlier quoted context omitted.

It also says the attribute of squeaking means it'll definitely float

That's actually pretty clever because if it squeaks, there is air inside. How many squeaking ducks have you come across that don't float?

You could call it clever or you could call it a spurious correlation.

Re: Gemini "duck" demo was not done in realtime or with voice

#256

Post by kaffeeringe » Thu, Dec 07, 2023, 9:28 PM UTC

jmathai wrote: post #105

modeless wrote: post #94

That's not the only thing wrong. Gemini makes a false statement in the video, serving as a great demonstration of how these models still outright lie so frequently, so casually, and so convincingly that you won't notice, even if you have a whole team of researchers and video editors reviewing the output. It's the single biggest problem with LLMs and Gemini isn't solving it. You simply can't rely on them when correctn…

This seems to be a common view among some folks. Personally, I'm impartial. Search or even asking other expert human beings are prone to provide incorrect results. I'm unsure where this expectation of 100% absolute correctness comes from. I'm sure there are use cases, but I assume it's the vast minority and most can tolerate larger than expected inaccuracies.

1. Hunans may also never be 100% - but it seems they are more often correct. 2. When AI is wrong it's often not only slighty off, but completely off the rails. 3. Humans often tell you when they are not sure. Even if it's only their tone. AI is always 100% convinced it's correct.

Re: Gemini "duck" demo was not done in realtime or with voice

#257

Post by modeless » Thu, Dec 07, 2023, 9:28 PM UTC

janalsncm wrote: post #237

modeless wrote: post #186

Earlier quoted context omitted.

I'm not a fan of current approaches here. "Chain of thought" or other approaches where the model does all its thinking using a literal internal monologue in text seem like a dead end. Humans do most of their thinking non-verbally and we need to figure out how to get these models to think non-verbally too. Unfortunately it seems that Gemini represents no progress in this direction.

The point of “verbalizing” the chain of thought isn’t that it’s the most effective method. And frankly I don’t think it matters that humans think non verbally. The goal isn’t to create a human in a box. Verbalizing the chain of thought allows us to audit the thought process, and also create further labels for training.

No, the point of verbalizing the chain of thought is that it's all we know how to do right now.

> And frankly I don’t think it matters that humans think non verbally

You're right, that's not the reason non-verbal is better, but it is evidence that non-verbal is probably better. I think the reason it's better is that language is extremely lossy and ambiguous, which makes a poor medium for reasoning and precise thinking. It would clearly be better to think without having to translate to language and back all the time.

Imagine you had to solve a complicated multi-step physics problem, but after every step of the solution process your short term memory was wiped and you had to read your entire notes so far as if they were someone else's before you could attempt the next step, like the guy from Memento. That's what I imagine being an LLM using CoT is like.

Re: Gemini "duck" demo was not done in realtime or with voice

#258

Post by mtrovo » Thu, Dec 07, 2023, 9:31 PM UTC

I guess a much better next step is to compare how GPT4V performs when asked similar prompts. Even if mostly staged this is very impressive to me, not much on the current tech but more on how much leverage Google has to win this race on the long run because of its hardware presence.

The more these models improve the more we will want less friction and faster interactions, this means that in the long term having to open an app and ask a question is not gonna fly compared to just pointing your phone camera to something, asking a question and getting an answer that's tailored to everything Google knows about you in real time.

Apple will most likely also roll their own in house solution for Siri instead of relying on an external company. This leaves OpenAI and the other small companies not just competing for the best models but also on how to put them in front of people in the first place and how to get access to their personal information.

Re: Gemini "duck" demo was not done in realtime or with voice

#259

Post by DonnyV » Thu, Dec 07, 2023, 9:32 PM UTC

This is so crazy. Google invented transformers which is the bases for all these models. How do they keep fumbling like this over and over. Google Docs created in 2006! Microsoft is eating their lunch. Google creates the ability to change VM's in place and makes a fully automated datacenter. Amazon and Microsoft are killing them in the cloud. Google has been working on self driving longer than anyone. Tesla is catching up and will most likely beat them.

The amount of fumbles is monumental.

Re: Gemini "duck" demo was not done in realtime or with voice

#260

Post by [deleted] » Thu, Dec 07, 2023, 9:33 PM UTC

modeless wrote: post #94

That's not the only thing wrong. Gemini makes a false statement in the video, serving as a great demonstration of how these models still outright lie so frequently, so casually, and so convincingly that you won't notice, even if you have a whole team of researchers and video editors reviewing the output. It's the single biggest problem with LLMs and Gemini isn't solving it. You simply can't rely on them when correctn…

[deleted]