Live data from Hacker News

Yishan Wong: "Google's Gemini issue is not about woke/DEI"

twitter.com

31–40 of 137 posts

Re: Yishan Wong: "Google's Gemini issue is not about woke/DEI"

#31
post #9

In a nutshell for those without a X account: the problem here isn't that Google's AI gave "woke" results. The big problem here is that even one of the biggest AI specialists in the world, Google, wasn't able to fully predict or control the output of its AI. If we ever create a truly powerful, superintelligent AI under the assumption that we'll be able to control it, this incident shows that even a minor mistake from…

I mean, in this case it seems to involve a staggering level of lack of QA.

Re: Yishan Wong: "Google's Gemini issue is not about woke/DEI"

#32
post #9

In a nutshell for those without a X account: the problem here isn't that Google's AI gave "woke" results. The big problem here is that even one of the biggest AI specialists in the world, Google, wasn't able to fully predict or control the output of its AI. If we ever create a truly powerful, superintelligent AI under the assumption that we'll be able to control it, this incident shows that even a minor mistake from…

>this incident shows that even a minor mistake from the AI builders could lead to the AI freeing itself from our control.

There's most likely a hidden prompt which tells Gemini to make people racially diverse. And Gemini complied perfectly. I can't call it "freeing itself from our control". It did exactly what was asked.

It's just a bug which was introduced by software engineers. No one says "Windows freed itself from our control" when BSOD happens. It's just a bug.

Re: Yishan Wong: "Google's Gemini issue is not about woke/DEI"

#33
post #32
post #9

In a nutshell for those without a X account: the problem here isn't that Google's AI gave "woke" results. The big problem here is that even one of the biggest AI specialists in the world, Google, wasn't able to fully predict or control the output of its AI. If we ever create a truly powerful, superintelligent AI under the assumption that we'll be able to control it, this incident shows that even a minor mistake from…

>this incident shows that even a minor mistake from the AI builders could lead to the AI freeing itself from our control. There's most likely a hidden prompt which tells Gemini to make people racially diverse. And Gemini complied perfectly. I can't call it "freeing itself from our control". It did exactly what was asked. It's just a bug which was introduced by software engineers. No one says "Windows freed itself fro…

This. We know this from the dumped System Prompt for Dall-e

Re: Yishan Wong: "Google's Gemini issue is not about woke/DEI"

#34
This is also a symptom of a corporation not having dedicated QA people. Sure, developers and product owners can check stuff, they can even read code and find bugs there. Amazing, we don't need QA then, right? Well, that's what we can sometimes get without QAs - an amazing code which actually works and works better than competitors. It just sometimes break in the areas not tested sufficiently or at all.

Re: Yishan Wong: "Google's Gemini issue is not about woke/DEI"

#35
post #33
post #32

Earlier quoted context omitted.

>this incident shows that even a minor mistake from the AI builders could lead to the AI freeing itself from our control. There's most likely a hidden prompt which tells Gemini to make people racially diverse. And Gemini complied perfectly. I can't call it "freeing itself from our control". It did exactly what was asked. It's just a bug which was introduced by software engineers. No one says "Windows freed itself fro…

This. We know this from the dumped System Prompt for Dall-e

Not to say, miscommunication/misunderstanding happens between people all the time. And software has already killed/injured people without any AI involved. Nothing specific to AI here. It's more about a lack of adequate QA.

Re: Yishan Wong: "Google's Gemini issue is not about woke/DEI"

#37
post #26

My personal opinion is that what he says is non-sense divagation. The problem of woke instructions like this is that it goes against the physical reality of the dataset. And it is the physical reality that does that we have what we have even if we don't like it. Such a case better show us the issue with the woke mindset that tries to bend the reality. You might want to ask the AI to only generate positive opinion, me…

The article addresses this very point, using the example of climate change: > it will be VERY plausible for an AI to simply conclude that it should proceed with the most expedient way to delete ~95% of humans. The elimination of ~95% of humans is a realistic way of "solving" climate change, without bending reality. It just so happens that this is not a morally acceptable solution to most of us. In other words, it is…

The example of climate change is sexy but I don't buy it as it is too overly simplified.

In my opinion, without woke and bending, the AI would probably reply "I don't know" like would a serious person.

Because, first you would have to define climate change and what is the problem, and what is the outcome that you would want.

Earth in itself does not care about what is going on. Similarly, there is no reason why a change of climate would be bad, it happened multiple time over the life of earth.

So, the request would probably be something like this: the current (or past century) climate was good for human life, so how could we do to restore or preserve that so that it stays favorable for us. As a side goal, we would like to preserve the existing variety of plants, trees and animals. As another side goal, we would like to preserve some landscape that are looking very nice. In this order.

Re: Yishan Wong: "Google's Gemini issue is not about woke/DEI"

#39
post #20

Earlier quoted context omitted.

The woke results were by design. This was not a mistake or unpredictable.

Are you contending that the black Wehrmacht soldiers, or black British kings eating watermelon, were anticipated or desired by Google staff? Even if you model them as the most extreme conceivable woke boogeymen, surely they wouldn't want a scenario where the diversity programming produced a result that they would consider more insulting to minorities than the one that would have been produced without it.

The Google staff responsible were clearly lacking imagination. I think next time Google should consider hiring a QA team consisting of teenagers from 4chan, I think they would excel at the job of figuring out ways to generate offensive content with the app.

Of course even better would be utilizing more balanced training data, so you don't need silly and badly working prompt engineering to force more diversity.

Post reply on HN