Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scientists Increasingly Can’t Explain How AI Works

vice.com

Post reply on HN
Print view View original

91–100 of 231 posts

Re: Scientists Increasingly Can’t Explain How AI Works

#91

Post by criddell »

dmichulke wrote: post #65
criddell wrote: post #42

Earlier quoted context omitted.

In the US, more than 100 people are going to die in their car today. If hardware and software flaws kill people at a lower rate than the shitty drivers they replace, then that's a win.

I agree with you. However, we can't really meaningfully quantify it a-priori. So we need a sufficiently big set of accidents and accident free rides to see when it's better. In addition we can't distinguish what driver would have been better or worse than the AI in a given situation. This opens the door for all sorts of lawsuits and we will end up with legal self-driving systems in almost ideal situations only. To no…

> To note, this is as much an engineering problem as it is societal and legislative.

Yeah, it definitely is. I think there are two things that should happen.

In the short term, producers of self-driving cars (or systems) should be given immunity from lawsuits related to crashes provided that the NHTSA (or some other authority) can verify that the deaths per passenger mile in their cars does not exceed the rate for traditional cars. Once the majority of passenger miles are in self-driven cars, revoke the immunity.

For the long term, software engineering should become a real engineering discipline and we require self-driving systems be signed off by licensed Professional Engineers. If those engineers ship software with bugs that lead to death or damage, those engineers can be held responsible, sued for malpractice, and have their license to practice revoked.

Re: Scientists Increasingly Can’t Explain How AI Works

#92

Post by mdp2021 »

saladtoes wrote: post #25

Explainability is not a given in many more traditional complex systems. Decisions are often an aggregation of a large number of signals, and one can often not conceive of a single intuitive explanation for the system's decisions. A lot is expected of AI systems today, from fairness (how do we even define that?) to universality. In my view we need to develop a practical understanding of what it means to build the syst…

> I would much rather have

False dichotomy. In fact, well understood systems must be more reliable.

Re: Scientists Increasingly Can’t Explain How AI Works

#93

Post by MichaelZuo »

impalallama wrote: post #78
fallingknife wrote: post #72

Earlier quoted context omitted.

Assuming that a human stereotype is untrue is just as biased as assuming it is true. If you judge a model by how well it confirms your previous bias, then the model tells you nothing you didn't already know.

Quoted post unavailable.

To be fair to the parent, prejudice means, according to the OED: preconceived opinion that is not based on reason or actual experience.

So preconceived opinions based on reason or actual experience do not count as prejudice.

Re: Scientists Increasingly Can’t Explain How AI Works

#94

Post by hcurtiss »

impalallama wrote: post #78
fallingknife wrote: post #72

Earlier quoted context omitted.

Assuming that a human stereotype is untrue is just as biased as assuming it is true. If you judge a model by how well it confirms your previous bias, then the model tells you nothing you didn't already know.

Quoted post unavailable.

This response is unnecessarily hostile. A machine stereotyping may have overwhelming accuracy. In that case, I would argue the machine is not wrong. It's our policy response to its outputs that may or may not be wrong. Changing the machine to disregard accurate descriptions of the real world only serves to bias the machine.

Re: Scientists Increasingly Can’t Explain How AI Works

#95

Post by belval »

mdp2021 wrote: post #87

Now for a small elephant which entered the room: the article recites that it would be > worth mentioning that the white-box / black-box terminology is in itself part of a long history of racially coded terms in science; researchers have pushed to change "blacklist" to "blocklist," for example Very plainly, the idea of "black-box" comes from the clear, basic and original, notion and experience that "in the dark, you c…

I guess it's the hill I'll die on, but these "white vs black" and "master slave" are inherently racist just make my head turn every time.

blacklist came from BEFORE slavery in America.

> According to the Henry Holt Encyclopedia of Word and Phrase Origins the word "blacklist" originated with a list England's King Charles II made of fifty-eight judges and court officers who sentenced his father, Charles I, to death in 1649. When Charles II was restored to the throne in 1660, thirteen of these regicides were executed and twenty-five sentenced to life imprisonment, while others escaped.

Thinking that blackbox is racist is approaching insane level of mind-bending "everything is about race".

Re: Scientists Increasingly Can’t Explain How AI Works

#96

Post by bobosha »

mdp2021 wrote: post #51
bobosha wrote: post #12

I think explainability is overrated (to use a Trumpian expression) & this fixation with coming up with explanations for inferences is a red herring. We cannot explain our own thoughts and actions and tend to ascribe logic & reason to many of our own actions, but it's almost always system-1 driven, for the most part. Correcting for edge-cases and unknown-unknowns is where we should focus our efforts methinks.

A very critical core human activity is that of reflection. You should explain the point about «correcting for edge-cases and unknown-unknowns», which may not be clear.

Thanks, but enough neuroscience studies have shown that our decisions are for the most part, on autopilot, qv: Kahneman, Tversky et al. All our "explanations" are post-hoc rationalization i.e. reality is the narrative we tell ourselves.

>You should explain the point about «correcting for edge-cases and unknown-unknowns», which may not be clear.

Our learning is also for the most part Hebbian, from childhood through adulthood. For example, a DUI might force one to rethink their transport after a night-out. Right now, some of the "shocking" predictions have a child-like brutal honesty about them. Just as a child is coached not to call that bad aunty "fatty", an AI can thus be trained to conform to societal norms.

tl;dr: i am advocating for a realtime continuous learning system

Re: Scientists Increasingly Can’t Explain How AI Works

#97

Post by lostlogin »

FartyMcFarter wrote: post #7

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

> Whenever you perform a massive amount of computation, you can get results that are nearly impossible to explain.

My thought on reading your comment was Douglas Adams and Deep Though’s 42.

Re: Scientists Increasingly Can’t Explain How AI Works

#99

Post by science4sail »

omginternets wrote: post #53
kriro wrote: post #40

I don't see a problem with not being able to explain how something works as long as it's not failing empirically. Scientists also cannot explain how humans work in detail and yet humans are "allowed" to do many things and make many decisions that cannot be fully explained. The question for me is...if I have an AI system that outperforms humans empirically, why do I need to understand how it works to use it? In fact i…

There's the obvious problem of knowing if and when it might suddenly start failing empirically.

Isn't that also a problem with humans? A human pilot might suddenly go insane and start murdering people (even though empirically, it's an improbable event).

https://en.wikipedia.org/wiki/Germanwings_Flight_9525

Re: Scientists Increasingly Can’t Explain How AI Works

#100

Post by joxel »

aliqot wrote: post #86
joxel wrote: post #71

Earlier quoted context omitted.

Why? How is it AI's fault that it can solve problems humans have proven unable to solve? And then you think it's AI's fault when we can't explain how it's solving the problems we can't solve? There's a reason we can't explain why a neural network detects it as a cat or dog, it's because we can't solve it ourselves, so the exact mathematical intuition is literally beyond our ability to describe (so far).

"How can a thousand monkeys solve what humans can't solve"

That is not analogous
Post reply on HN