Scientists Increasingly Can’t Explain How AI Works • Hacker News Board

Re: Scientists Increasingly Can’t Explain How AI Works

#131

Post by JacobiX » Wed, Nov 02, 2022, 3:38 PM UTC

visarga wrote: post #107

kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

> you can't explain what's going on inside that black box Do you have a better solution for speech recognition? We all know how well speech recognition works in reality, and we know language models can accomplish more complex tasks than setting your music and lights. These models are not state of the art, they are cheap versions for scaling up to millions of users. It's sad but we rarely get to see SOTA in a product.

The fact that a machine learning model can solve a seemingly hard task A does not automatically translate to solving a lesser hard task B. Conversational AI that understand the intent remains a really hard problem even if speech recognition for English is basically solved.

Re: Scientists Increasingly Can’t Explain How AI Works

#132

Post by UncleMeat » Wed, Nov 02, 2022, 3:42 PM UTC

FartyMcFarter wrote: post #7

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

Older ML techniques are explainable in nice ways. With something like logistic regression you get a very nice "this feature contributed this much to the final classification" breakdown. It just turns out that logistic regression is considerably less effective than RNNs, which have all these synthetic features within the inner layers that aren't easily meaningful to humans.

Re: Scientists Increasingly Can’t Explain How AI Works

#133

Post by alex_suzuki » Wed, Nov 02, 2022, 3:42 PM UTC

Skynet… and so it begins.

Re: Scientists Increasingly Can’t Explain How AI Works

#134

Post by endtime » Wed, Nov 02, 2022, 3:43 PM UTC

kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

I worked on Google Assistant for ~four years. The NLP is important, but it's still just a slice of making the thing work. And while there's some cutting edge stuff going on there, it's not so general or mysterious that it can't be understood or controlled, and I don't think it's making things worse.

Educated guess: your music issues can be blamed on a ranking subroutine owned by a "media" or "music" team, not on "black box AI".

Re: Scientists Increasingly Can’t Explain How AI Works

#135

Post by cultofmetatron » Wed, Nov 02, 2022, 3:45 PM UTC

if the human brain was simple to understand, we wouldn't be smart enough to understand it.

Is it really surprising that as AI approaches our level of intelligence, the underlying interactions become more and more of a black box?

Re: Scientists Increasingly Can’t Explain How AI Works

#136

Post by tehsauce » Wed, Nov 02, 2022, 3:47 PM UTC

A system only being able to perform tasks that you understand is a huge limitation. This will only continue as civilization transitions to silicon based life.

Re: Scientists Increasingly Can’t Explain How AI Works

#137

Post by habitue » Wed, Nov 02, 2022, 3:47 PM UTC

I think it's very possible that we won't have good explanations for things that are this complicated and that change this quickly. I.e. massive matrices of numbers that are trained and retrained regularly.

Humans have a small amount of working memory, and anything bigger has to be knitted together from pointers to concepts stored in long term memory. Long term memory takes a while to construct.

Let's say you take one single forward pass of a neutral network to output a single image. To really understand what's happening, you'd have to study that network for years, tracing which parts of the network's matrices influence which parts of the output image.

How long would it take to really build a deep intuition about that single network, or even that single forward pass? I'd think as a starting point it would take as long to learn as a language, and probably much longer.

Re: Scientists Increasingly Can’t Explain How AI Works

#138

Post by magic_hamster » Wed, Nov 02, 2022, 3:48 PM UTC

kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

I must be in the minority but I really dislike voice commands. Every time I try it I feel stupid talking to appliances.

But even if I didn't mind talking to lightbulbs and thermostats it would still bother me for several reasons: I don't like being listened to by these corporations, I don't like exposing my electronics to the internet, and I still don't see how any of this stuff is better than flicking a switch.

Re: Scientists Increasingly Can’t Explain How AI Works

#139

Post by jjk166 » Wed, Nov 02, 2022, 3:48 PM UTC

That's the point. If we had good and intuitive models of how a system works, we could simply write a program to calculate whatever result we were looking for. The purpose of AI is to look at data and find patterns that the human mind struggles to pick up thus allowing us to make accurate predictions without understanding the underlying rules. In the early days of AI when we were basically just practicing, we applied them to problems that we knew the answers to because we understood the underlying rules, and thus could see easily how close AIs got to accurately learning these rules. But now as AI has graduated to the level where it can be used for real world applications, of course now it is successfully doing its job of figuring out relations which are not only difficult to spot but also difficult to turn into an intuitive narrative.

Re: Scientists Increasingly Can’t Explain How AI Works

#140

Post by visarga » Wed, Nov 02, 2022, 3:48 PM UTC

naasking wrote: post #105

impalallama wrote: post #78

Earlier quoted context omitted.

Quoted post unavailable.

Setting aside the rudeness, you basically just agreed with the parent. They said people don't want an AI with no bias, we want an AI that mirrors own biases. Your example of loans is perfectly in line with this: we have a social bias against racism and racial profiling, and we want our AIs to reflect that bias. Purely on the question of risk assessment, race must meaningfully impact a loan risk or the AI wouldn't dev…

I think people are conflating the technical sense of "bias" to the social sense. Technically, a biased model is one that overfits the training data and has poor generalisation. Ethically, a biased model would be one that reinforces discrimination.