Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI behavior guardrails should be public

twitter.com

Post reply on HN
Print view View original

121–130 of 350 posts

Re: AI behavior guardrails should be public

#121

Post by mp05 »

DebtDeflation wrote: post #90
dekhn wrote: post #12

I strongly suspect Google tried really, really hard here to overcome the criticism is got with previous image recognition models saying that black people looked like gorillas. I am not really sure what I would want out of an image generation system, but I think Google's system probably went too far in trying to incorporate diversity in image generation.

Surely there is a middle ground. "Generate a scene of a group of friends enjoying lunch in the park." -> Totally expect racial and gender diversity in the output. "Generate a scene of 17th century kings of Scotland playing golf." -> The result should not be a bunch of black men and Asian women dressed up as Scottish kings, it should be a bunch of white guys.

> "Generate a scene of a group of friends enjoying lunch in the park." -> Totally expect racial and gender diversity in the output.

I'd err on the side of "not unexpected". A group of friends in a park in Tokyo is probably not very diverse, but it's not outside of the realm of possibility. Only white men were golfing Scottish kings if we're talking strictly about reality and reflecting it properly.

Re: AI behavior guardrails should be public

#122

Post by photoGrant »

CuriouslyC wrote: post #114
photoGrant wrote: post #98

Earlier quoted context omitted.

No, compute is keys to the kingdom. The rest are assets and ammunition. You out-compute your enemy, you out-compute your competition. That's the race. The data is part of the problem, not the root. These companies are silo'ing the worlds resources. GPU, Finance, Information. Those combined are the weapon. You make your competition starve in the dust. These companies are pure evil pushing an agenda of pure evil. OpenA…

Compute lets you fuck up a lot when trying to build a model, but you need data to do anything worth fucking up in the first place, and if you have 20% of the compute but you fuck up 1/5th as much you're doing fine. Meta/OpenAI/Google can fuck up a lot because of all their compute, but ultimately we learn from that as the scientists doing the research at those companies would instantly bail if they couldn't publish pa…

I never said each of these exist in a vacuum. It is the collation of all that is the danger. This isn't democratic. This is companies now toying with governmental ideologies.

Re: AI behavior guardrails should be public

#123

Post by ankit219 »

DebtDeflation wrote: post #90
dekhn wrote: post #12

I strongly suspect Google tried really, really hard here to overcome the criticism is got with previous image recognition models saying that black people looked like gorillas. I am not really sure what I would want out of an image generation system, but I think Google's system probably went too far in trying to incorporate diversity in image generation.

Surely there is a middle ground. "Generate a scene of a group of friends enjoying lunch in the park." -> Totally expect racial and gender diversity in the output. "Generate a scene of 17th century kings of Scotland playing golf." -> The result should not be a bunch of black men and Asian women dressed up as Scottish kings, it should be a bunch of white guys.

The focus for alignment is to avoid bad PR specifically the kind of headlines written by major media houses like NYT, WSJ, WaPo. You could imagine the headlines like "Google's AI produced a non-diverse output on occasions" when a researcher/journalist is trying too hard to get the model to produce that. The hit on Google is far bigger than say on Midjourney or even Open AI till now (I suspect future models will be more nerfed than what they are now)

For the cases you mentioned, initially those were the examples. It gets tricky during red teaming where they internally try out extreme prompts and then align the model for any kind of prompt which has a suspect output. You train the model first, then figure out the issues, and align the model using "correct" examples to fix those issues. They either went to extreme levels doing that or did not test it on initial correct prompts post alignment.

Re: AI behavior guardrails should be public

#124

Post by sho_hn »

DebtDeflation wrote: post #90
dekhn wrote: post #12

I strongly suspect Google tried really, really hard here to overcome the criticism is got with previous image recognition models saying that black people looked like gorillas. I am not really sure what I would want out of an image generation system, but I think Google's system probably went too far in trying to incorporate diversity in image generation.

Surely there is a middle ground. "Generate a scene of a group of friends enjoying lunch in the park." -> Totally expect racial and gender diversity in the output. "Generate a scene of 17th century kings of Scotland playing golf." -> The result should not be a bunch of black men and Asian women dressed up as Scottish kings, it should be a bunch of white guys.

Quite reminded of this episode: https://www.eurogamer.net/kingdom-come-deliverance-review (black representation in a video game about 15th century Bohemia; it was quite the controversy)

Re: AI behavior guardrails should be public

#125

Post by [deleted] »

123yawaworht456 wrote: post #11
Jensson wrote: post #2

They know that people would be up in arms if it generated white men when you asked for black women so they went the safe route, but we need to show that the current result shouldn't be acceptable either.

the models are perfectly capable of generating exactly what they're told to. instead, they covertly modify the prompts to make every request imaginable represent the human menagerie we're supposed to live in. the results are hilarious. https://i.4cdn.org/g/1708514880730978.png

[deleted]

Re: AI behavior guardrails should be public

#126

Post by photoGrant »

CuriouslyC wrote: post #120
photoGrant wrote: post #116

Earlier quoted context omitted.

No, ignoring laws and stealing data to increase your Castle's MOAT is the win. Compute isn't an open source solvable problem. I can't DirtyPCB's an A100 Making the argument open source is the answer is an agenda of making your competition spin wheels.

You're on a thread about how people are lambasting big money AI for being garbage, and producing inferior results to OSS tools you can run on consumer GPUS, tell me again how unbeatable google/other big tech players are.

I've been part of the advertising and marketing world for a lot of these companies for a decade plus, I've helped them sell bullshit. I've also been at the start of the AI journey, I've downloaded and checked local models of all promises and variances.

To say they're better than the compute that OpenAI or Google are throwing at the problem is just plain wrong.

I left the ad industry the moment I realised my skills and talents are better used informing people than lying to them.

This thread is not at all comparing the ethical issues of AI with local anything. You're conflating your solution with another problem.

Re: AI behavior guardrails should be public

#127

Post by _heimdall »

michaelt wrote: post #51
dekhn wrote: post #12

I strongly suspect Google tried really, really hard here to overcome the criticism is got with previous image recognition models saying that black people looked like gorillas. I am not really sure what I would want out of an image generation system, but I think Google's system probably went too far in trying to incorporate diversity in image generation.

As well as that, I suspect the major AI companies are fearful of generating images of real people - presumably not wanting to be involved with people generating fake images of "Donald Trump rescuing wildfire victims" or "Donald Trump fighting cops". Their efforts to add diversity would have been a lot more subtle if, when you asked for images of "British Politician" the images were recognisably Rishi Sunak, Liz Truss…

We're honestly just seeing generative algorithms failing at diversity initiatives as badly as humans for.

Forcing diversity into a system is an extremely tough, if not impossible, challenge. Initiatives have to be driven my goals and metrics, meaning we have to boil diversity down to a specific list of quantitative metrics. Things will always be missed when our best tool to tackle a moral or noble goal is to boil a complex spectrum of qualitative data to a subset of measurable numbers.

Re: AI behavior guardrails should be public

#128

Post by sct202 »

I'm very curious what geography the team who wrote this guardrail came from and the wording they used. It seems to bias heavily towards generating South Asian (especially South Asian women) and Black people. Latinos are basically never generated which would be a huge oversight if they were based in the USA, but stereotypical Native American looking in the distance and East Asians sometimes pop up in the examples people are showing.

Re: AI behavior guardrails should be public

#129

Post by josephg »

CuriouslyC wrote: post #109
photoGrant wrote: post #54

Earlier quoted context omitted.

Remind yourself we're discussing censorship, misinformation, inability to define or source truth and we're concerned on Day 1 about the results of image gen being controlled by a for profit single entity with incentives that focus solely on business and not humanity... Where do we go from here? Things will magically get better on their own? Businesses will align with humanity and morals, not their investors? This is…

The ridiculous degree of PC alignment of corporate models is the thing that's going to let open source win. Few people use bing/dall-e, but if OpenAI had made dall-e more available and hadn't put ridiculous guardrails on it, stable diffusion would be a footnote at this point. Instead, dall-e is a joke and people who make art use stable diffusion, with casuals who just want some pretty looking pictures using midjourne…

Don’t count out Adobe Firefly. I wouldn’t be surprised if it’s used more than all the other image gen models combined.

Re: AI behavior guardrails should be public

#130

Post by CuriouslyC »

photoGrant wrote: post #126
CuriouslyC wrote: post #120

Earlier quoted context omitted.

You're on a thread about how people are lambasting big money AI for being garbage, and producing inferior results to OSS tools you can run on consumer GPUS, tell me again how unbeatable google/other big tech players are.

I've been part of the advertising and marketing world for a lot of these companies for a decade plus, I've helped them sell bullshit. I've also been at the start of the AI journey, I've downloaded and checked local models of all promises and variances. To say they're better than the compute that OpenAI or Google are throwing at the problem is just plain wrong. I left the ad industry the moment I realised my skills an…

Is a $5 can opener better than a $2000 telescope at opening cans? Yes. Is stable diffusion better at producing finished art, by virtue of not being closed off and DEI'd to oblivion so that it can actually be incorporated into workflows? Emphatically yes.

It doesn't matter how fancy your engineering is and how much money you have if you're too stupid to build the right product.

As for this being written nonsense, that's the sort of thing someone who couldn't find an easy way to win an argument and was bitter about the fact would say.

Post reply on HN