Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is artificial intelligence permanently inscrutable?

nautil.us

Post reply on HN
Print view View original

51–60 of 69 posts

Re: Is artificial intelligence permanently inscrutable?

#51

Post by colllectorof »

hyperion2010 wrote: post #38

This is not just an issue for neural nets, but also for brains. Our interpretations of our own actions should always be considered posthoc rationalizations in the absence of some falsifiable experiment being conducted to demonstrate the validity of the interpretation. Human brains are excellent at creating a coherent story about the world they experience based on the data at hand, thus we suffer the same kinds of iss…

You're hand-waving a real problem by asserting rather misanthropic ideas with scant justification.

Show some child a cat and ask why it's not a crocodile. They will be able to explain it through the differences in shape, color, behavior and other features of those animals. Whether you consider it post-hoc is unimportant. The explanation is still real and relevant.

So people can explain their deliberate reasoning. Also, reason about reasoning. If they truly can't, it's called gut feeling and no one rational trusts those for solving complex problems with important outcomes. Especially at scale.

Neural networks, at least right now, have no capabilities of this sort. There are some attempts at visualization, but they are inherently limited, because they must make assumptions about the domain of the problem.

Re: Is artificial intelligence permanently inscrutable?

#52

Post by empath75 »

colllectorof wrote: post #51
hyperion2010 wrote: post #38

This is not just an issue for neural nets, but also for brains. Our interpretations of our own actions should always be considered posthoc rationalizations in the absence of some falsifiable experiment being conducted to demonstrate the validity of the interpretation. Human brains are excellent at creating a coherent story about the world they experience based on the data at hand, thus we suffer the same kinds of iss…

You're hand-waving a real problem by asserting rather misanthropic ideas with scant justification. Show some child a cat and ask why it's not a crocodile. They will be able to explain it through the differences in shape, color, behavior and other features of those animals. Whether you consider it post-hoc is unimportant. The explanation is still real and relevant. So people can explain their deliberate reasoning. Als…

Most actions people take are not the result of deliberate reasoning.

You may be able to explain why a cat is different than a dog, but your brain doesn't go through that categorization process before recognizing a cat.

Re: Is artificial intelligence permanently inscrutable?

#53

Post by hyperion2010 »

colllectorof wrote: post #51
hyperion2010 wrote: post #38

This is not just an issue for neural nets, but also for brains. Our interpretations of our own actions should always be considered posthoc rationalizations in the absence of some falsifiable experiment being conducted to demonstrate the validity of the interpretation. Human brains are excellent at creating a coherent story about the world they experience based on the data at hand, thus we suffer the same kinds of iss…

You're hand-waving a real problem by asserting rather misanthropic ideas with scant justification. Show some child a cat and ask why it's not a crocodile. They will be able to explain it through the differences in shape, color, behavior and other features of those animals. Whether you consider it post-hoc is unimportant. The explanation is still real and relevant. So people can explain their deliberate reasoning. Als…

Scant justification written here certainly. Your example of the child is not a good one in this context because the child can only do it based on knowledge or based on 'they look different.' Well turns out black people and white people 'look different' but these days we now know that we are all members of the same species.

The examples I am more worried about are not knowledge based (or judgement based). The ones I am worried about are the stories we tell about the _actions_ we take, why we choose A instead of B. Answering these requires us to introspect our own behavior and that is exceptionally difficult. Take for example a question like "how did you know it was safe to cross the street?" Many people will tell you that they 'looked both ways and saw nothing was coming' but if you have them wear earplugs they will get run over because it was really the fact that they were listening for the cars. Or say you meet someone and you decide you don't like them, you can't really go back an undo that, but it might actually have been the case that you didn't like the perfume they were wearing because it was the same one that your ex used to wear, and it had nothing to do with the person themselves. Virtually no one can catch all the cases like this because we are trapped by our own brains and our own blind spots. Buying your brain's bullshit without attempting some independent verification (eg by asking someone for another perspective or doing little experiments on yourself) is a exceptionally common, and difficult problem to overcome.

Re: Is artificial intelligence permanently inscrutable?

#54

Post by nurettin »

Adrian Thompson's 1996 paper was about Genetic Algorithms. A poor overfitting example considering the whole article is prominently about Artificial Neural Networks. Thompson's FPGA components were trained at room temperature and the creatures were unable to function well when the temperature deviates too much from 10 deg. C.

Re: Is artificial intelligence permanently inscrutable?

#55

Post by unabst »

Isn't this all simply about correlation vs causation? Machine learning can find strong correlations and we can make predictions based on those correlations, but at the end of the day, the machine knows nothing about what is causing any of it, and hence is "inscrutable".

So it is up to us to fill the gap in our understanding because that is what machine learning ultimately says about the subject. It tells us what we don't know. If we knew all about the subject, our predictions would match the predictions of the machine because there is only one reality we're both observing. But if there is any gap, then the machine is telling us what we don't know, not what it (of all things) knows. It's just crunching numbers. It doesn't "know" anything.

Re: Is artificial intelligence permanently inscrutable?

#56

Post by AndrewKemendo »

When I try to explain neural nets (specifically in vision systems) to people I basically explain how you take inputs in the form of images, label pixels/pixel groups in the images with what you want them to output in the future, and then do that thousands of times and continue to test the results.

Critically though, I will say something to the effect of "but if you try and break the net open and see how this specific net came to it's result, it will look like spaghetti"

So it's a roundabout way of saying "junk in; junk out." That holds true for any learning system, including human animals. The thought process of humans is inscrutable thus far, and I think that future computing will be similarly inscrutable if we do it correctly.

Re: Is artificial intelligence permanently inscrutable?

#57

Post by EGreg »

empath75 wrote: post #52
colllectorof wrote: post #51

Earlier quoted context omitted.

You're hand-waving a real problem by asserting rather misanthropic ideas with scant justification. Show some child a cat and ask why it's not a crocodile. They will be able to explain it through the differences in shape, color, behavior and other features of those animals. Whether you consider it post-hoc is unimportant. The explanation is still real and relevant. So people can explain their deliberate reasoning. Als…

Most actions people take are not the result of deliberate reasoning. You may be able to explain why a cat is different than a dog, but your brain doesn't go through that categorization process before recognizing a cat.

How do you know?

Re: Is artificial intelligence permanently inscrutable?

#58

Post by dharma1 »

The human brain also does massive dimensionality reduction on very large amounts of data, and a lot of unconscious processing, with much of it being beyond our capabilities of conscious introspection.

I think eventually, within a couple of decades, we will have AI that correlates well enough with human thought process, and has enough knowledge of the world, to be able to introspect and explain in various levels of detail, in natural language, images and other human readable constructs, why it has reached a certain conclusion. And we will be able to experimentally verify those explanations.

Re: Is artificial intelligence permanently inscrutable?

#59

Post by rbanffy »

EGreg wrote: post #57
empath75 wrote: post #52

Earlier quoted context omitted.

Most actions people take are not the result of deliberate reasoning. You may be able to explain why a cat is different than a dog, but your brain doesn't go through that categorization process before recognizing a cat.

How do you know?

The intermediate steps don't usually bubble up to conscious levels.

Sometimes I "know" a design is bad. Explaining why to others requires stepping through a process that May or may not be the one my brain used to make that approach "feel" wrong.

Re: Is artificial intelligence permanently inscrutable?

#60

Post by csydas »

empath75 wrote: post #52
colllectorof wrote: post #51

Earlier quoted context omitted.

You're hand-waving a real problem by asserting rather misanthropic ideas with scant justification. Show some child a cat and ask why it's not a crocodile. They will be able to explain it through the differences in shape, color, behavior and other features of those animals. Whether you consider it post-hoc is unimportant. The explanation is still real and relevant. So people can explain their deliberate reasoning. Als…

Most actions people take are not the result of deliberate reasoning. You may be able to explain why a cat is different than a dog, but your brain doesn't go through that categorization process before recognizing a cat.

Eh, I disagree, since a lot of times we rely heavily on visual cues to make assessments and determinations, and are often wrong just because of poor vision.

Take for example persons with androgynous features and the difficulty some people have with identifying them. Children are particularly useful for this insight since they are unabashedly curious and inquisitive, and it's not uncommon to hear a child blurt out "is that a boy or a girl?"; it would seem to me that their categorization process, even if not refined, gets goofed by androgyny; something as simple as hair length or facial hair greatly influences a child's ability to easily discern if persno is male or female.

At some level there is a categorization process that seems to happen with humans when it comes to recognition, we just assign high confidence to certain factors. In the case of children, it seems to be they look at person and use common factors like hair length, facial hair, and body shape to determine gender. The sound of one's voice also helps, but this usually isn't something we can pick up as easily or from afar in public places. As an adult, we probably have more refined points, but they are essentially the same as a child's point of view. Instead of "body shape" adults look for specific facial features, size of breasts, walking gait, style of clothing, hair style, etc. These are all just granular differences of how a child understands it.

This gets into personal experience, but when I was in college, I had long hair and really fine skin/facial features. All the time I would hear little kids ask "why does he look like a girl?" or "why does that girl have a beard?". I had no issue with their confusion, they were just curious kids. Something confused their understanding, and once it was explained "sometimes men have long hair", they just sort of accepted it.

Post reply on HN