Notes on AI Bias • Hacker News Board

Re: Notes on AI Bias

#121

Post by btilly » Thu, Apr 25, 2019, 9:05 PM UTC

Chris2048 wrote: post #117

btilly wrote: post #99

Earlier quoted context omitted.

One major problem. The parole software was NOT being fed data for "will this person commit another crime". It was being fed data for, "will this person be a suspect for another crime". The significant difference is that selective enforcement biases the data that it was trained on. Said selective enforcement has multiple causes, including the fact that heavier patrolling in black neighborhoods makes catching crimes mo…

Which represents ground truth better? arrest records, or survey results?

For this? Probably survey results. Particularly https://nsduhweb.rti.org/respweb/homepage.cfm.

Re: Notes on AI Bias

#122

Post by Bartweiss » Thu, Apr 25, 2019, 9:11 PM UTC

john-radio wrote: post #74

Bartweiss wrote: post #58

Earlier quoted context omitted.

That intern story is excellent; I'm adding it to my bank of "weird AI tricks" like pausing Tetris to avoid losing. More topically, you're quite right to object to that Amazon reference. As far as I can tell, the real story is even worse than mislabeling. Amazon devs wanted a system to spot candidates in resume banks, so they trained it to recognize resumes similar to the ones submitted to Amazon in the past. The enti…

> That intern story is excellent; I'm adding it to my bank of "weird AI tricks" like pausing Tetris to avoid losing. Post your bank! Let's be like Magnus Carlson and occasionally ask ourselves, "What would DeepMind do?"

Oh man, good question. I'm always up for swapping these stories. A lot of these came from a paper on weird AI tricks, and resulting best-of list on a blog collecting these stories.[1][2] Suffice to say, the people who think the orthogonality thesis is a weird hypothetical aren't keeping up with the state of things.

- The aforementioned Tetris story: an undirected learner was set to maximize score at Tetris learned normal play techniques, but also learned to pause the game immediately before losing so that the score wouldn't "decline" at game over.

- In the same vein as interns quitting, proxy detection of all sorts. Identify "field with sheep" by finding green fields with grey skies, or letting heuristics like "humans pick up dogs and cats" override correct identifications. (It's a goat until you pick it up, then it's a dog!)

- An agent playing Q*bert found a known bug for infinite lives, then escalated to an unknown bug which disabled the game while overflowing the score counter.

- Agents in a physics sim tasked with jumping as high as possible instead learned to 'fly' by abusing collision detection bugs, hitting themselves in ways that created upward momentum.

- Another "maximize jump height" task demonstrated that "highest" is an extremely fuzzy term. Initially measured by highest point, they became incredible tall. Measured by lowest point, they stayed tall and grew topheavy to 'kick' their base upwards.

- Number-handling bugs of all kinds. In one case, small twitches led to floating-point errors that created energy. In another, a "minimize force" task got solved by maximizing force and triggering integer wraparound.

My personal favorite is an adversarial bug. An agent playing tic-tac-toe on an infinite grid with a time limit submitted extremely remote moves which caused timeouts/crashes in any agent that tried to model the full board.

[1] https://arxiv.org/pdf/1803.03453.pdf

[2] https://aiweirdness.com/post/172894792687/when-algorithms-su...

Re: Notes on AI Bias

#123

Post by meheleventyone » Fri, Apr 26, 2019, 9:52 AM UTC

Chris2048 wrote: post #119

meheleventyone wrote: post #118

Earlier quoted context omitted.

Those were rhetorical questions I fully understand what the point of the example was.

but then it doesn't encode assumptions. Metaphors are supposed to be imperfect. "bias" and "reality" can equally cover for model simplicity.

Of course it encodes assumptions whether imperfect or not.

Re: Notes on AI Bias

#124

Post by Chris2048 » Fri, Apr 26, 2019, 5:06 PM UTC

meheleventyone wrote: post #123

Chris2048 wrote: post #119

Earlier quoted context omitted.

but then it doesn't encode assumptions. Metaphors are supposed to be imperfect. "bias" and "reality" can equally cover for model simplicity.

Of course it encodes assumptions whether imperfect or not.

Does a water-pipe analogy for electrical current encode the assumption that electricity is made of water?

Re: Notes on AI Bias

#125

Post by FakeComments » Sat, Apr 27, 2019, 3:17 AM UTC

dang wrote: post #107

FakeComments wrote: post #82

Earlier quoted context omitted.

It’s an absolute nonsense opinion, because there are obviously cultural beliefs that have an impact on work performance, particularly over a lifetime — eg, beliefs in honesty, hard work, and obtaining education. Different cultures place different values on these things, and it leads to different outcomes — even just looking at the impact of cultural drift over time, among a single people. I also find it absolutely ab…

If you keep doing ideological flamewars on HN we will ban you. Also, if you keep doing personal attacks on HN we will ban you. Please review https://news.ycombinator.com/newsguidelines.html and follow the rules when posting here from now on.

Again, I wish you would admit to ideological censorship:

You’re simply using etiquette as a guise for people who don’t want to have an honest discussion about the relevant philosophy (or in this case, the actual logic involved in the inference) to shut it down in favor of group think.

This problem is so widespread among the tech community, Congress is talking about it.

Just look at how fast this was downvoted and flagged — the point is to silence wrongthink.

Censorship for etiquette has always been used as a tool of political coercion — and I simply don’t take your vague claims I broke decorum, which always target one side of a sensitive issue, seriously.

You can absolutely use your power in your fiefdom; but you can’t use it to make me not talk honestly about issues — only remove me.

You’re just a political thug, who doesn’t like honest, blunt language.

Show me.

Ed:

There’s hundreds of comments like this without a peep, because they’re rightthink:

https://news.ycombinator.com/item?id=19763553

Re: Notes on AI Bias

#126

Post by dang » Sat, Apr 27, 2019, 5:07 AM UTC

FakeComments wrote: post #125

dang wrote: post #107

Earlier quoted context omitted.

If you keep doing ideological flamewars on HN we will ban you. Also, if you keep doing personal attacks on HN we will ban you. Please review https://news.ycombinator.com/newsguidelines.html and follow the rules when posting here from now on.

Again, I wish you would admit to ideological censorship: You’re simply using etiquette as a guise for people who don’t want to have an honest discussion about the relevant philosophy (or in this case, the actual logic involved in the inference) to shut it down in favor of group think. This problem is so widespread among the tech community, Congress is talking about it. Just look at how fast this was downvoted and fla…

"Censorshop for etiquette" is not even close to being the active ingredient here. I hear that it looks that way to you, but what's actually going on when we moderate threads is so different, that I wonder how to convey the difference. Any ideas?

Let me put it this way: I wonder how to convince you how little we give a shit about decorum. What a miserable life it would be to spend one's days enforcing something that...limited. Such a concept appears nowhere in the site guidelines or in its moderation history.

It's pretty striking to hear "You’re just a political thug", and I can certainly understand why your comments have so much force in them if you feel that way. When you say we "always target one side of a sensitive issue", though, that's so factually inaccurate that I'm not sure what else I can do that would help.