I think explainability is overrated (to use a Trumpian expression) & this fixation with coming up with explanations for inferences is a red herring. We cannot explain our own thoughts and actions and tend to ascribe logic & reason to many of our own actions, but it's almost always system-1 driven, for the most part. Correcting for edge-cases and unknown-unknowns is where we should focus our efforts methinks.
You can't explain your own thoughts and actions??? I'm sorry, but that's part of makes a valuable team mate or partner. This is not fundamentally human, this is trained behaviour. I am very well capable of expressing my thougts and explaining actions resulting from these thoughts.
To me the problem of AI is not how it works but how reliable it works. Do we have a way to test AI to prevent corner cases which could lead to catastrophic results?
What you're saying is (kinda) my response when people ask me about self-driving cars: How does the car deal with object X on the road? For X=piano, a stack of solar panels, a tank, an airplane, a pile of stones ... No one knows and since the size of the set of X is infinite, no one can appropriately train for it either. That's why we can't have self-driving cars without a general understanding of what objects are and…
In the US, more than 100 people are going to die in their car today. If hardware and software flaws kill people at a lower rate than the shitty drivers they replace, then that's a win.
> anything new It is nothing new. It is called (at least) "the problem of transparency". The chess context is probably not the best, because many systems allow a lengthy complex explanation of the response (I cannot remember now the exact workings of Deep Blue - it has been a while last time I met the full info). It is a real problem in general, because we may not just want responses but we may want " to learn someth…
In 1963 Marvin Minsky called it “The Credit Assignment Problem” — which, among a multitude of variables, were most important in solving an AI task?
There is a somewhat widespread implicit assumption that having sufficient intelligence to develop increasingly sophisticated machine learning implies that we also have sufficient intelligence to develop an “intuitive” understanding of how it works. I think that assumption is totally wrong, and trying to reconcile the two is probably a distraction and a waste of time. Evolution led to human intelligence just fine on i…
What people are really saying, but never will admit, is that they want AI to mirror their own biases, not to have none. That's the reason they are "cleaning" the input data.
The history of scientific development is one of finding patterns in data (think Kepler's studies of Tycho Brahe's accurate astronomical observations) - yes, AI excels at that - but then expressing those patterns in terms of simple mathematical equations: orbits are elliptical to a first approximation (not counting other graviational bodies than the sun and a planet), and the sun->planet vector sweeps out equal areas…
but what if it spat out einsteins equations, and that it somehow magically works but the people looking at that output could not imagine the "explanation"?
Wouldn't that look exactly like what AI we have today - able to make patterns or predictinos, but we cannot interpret the AI's "formula".
> Black box models can be extremely powerful, which is how many scientists and companies justify sacrificing explainability for accuracy. > AI systems have been used for autonomous cars, customer service chatbots, and diagnosing disease, and have the power to perform some tasks better than humans can. For example, a machine that is capable of remembering one trillion items, such as digits, letters, and words, versus…
> suspect this article was written by AI. Or a hack journalist.
A friend of mine did not belive that the AI based astroturfing problem is real, he though he would tell apart human tweets / letters / phonecalls vs bots. But have you seen some of the dumb shit real people post? the overlap between mental people and bad ai is huge
> Black box models can be extremely powerful, which is how many scientists and companies justify sacrificing explainability for accuracy. > AI systems have been used for autonomous cars, customer service chatbots, and diagnosing disease, and have the power to perform some tasks better than humans can. For example, a machine that is capable of remembering one trillion items, such as digits, letters, and words, versus…
> suspect this article was written by AI. Or a hack journalist. A friend of mine did not belive that the AI based astroturfing problem is real, he though he would tell apart human tweets / letters / phonecalls vs bots. But have you seen some of the dumb shit real people post? the overlap between mental people and bad ai is huge
I'm not sure how many people have messed around with the Character.ai beta but personally I'm awed that it does some of the things it does. It predictably fails at times and makes some odd mistakes, but at other times it insightfully points things out that I hadn't considered and explains itself quite eloquently when prompted. It's a lot better discussing literature and philosophy than science, though... and a compulsive liar. But a human editor willing to paper over the inconsistencies can definitely churn out articles in a fraction of the time with language models like this. And in regular conversation it's easily up there with, or surpasses, your average internet troll (which, as you pointed out, isn't exactly a high bar).
There is a somewhat widespread implicit assumption that having sufficient intelligence to develop increasingly sophisticated machine learning implies that we also have sufficient intelligence to develop an “intuitive” understanding of how it works. I think that assumption is totally wrong, and trying to reconcile the two is probably a distraction and a waste of time. Evolution led to human intelligence just fine on i…
What people are really saying, but never will admit, is that they want AI to mirror their own biases, not to have none. That's the reason they are "cleaning" the input data.
Yes and no. Yes, because I agree with what you said. No, because it assumes self-driving has to rely purely on normal vision/cameras, which is just garbage². Think LiDAR and other such means to detect obstacles, but things like that are (currently) pretty expensive, so there is the attempt to just do it with (more or less) normal cameras, which leads to what you said (and I fully agree with that). Aside of the fact t…
LiDAR can't tell you mass of an object as it is headed towards your vehicle. Imagine you're driving down the road and you see: 1.) A kid throw a rubber playground ball into the road and at your windshield. 2.) A kid heave a bowling ball into the road and at your windshield. How would you react in each case? How do you distinguish between the two? As a kid is heaving the bowling ball you can tell that it has a lot mor…
Same for both: brake. I don't want either hitting my windshield. The kickball may be bouncy, sure, but it may have gotten rocks stuck to it, which can create a point stress and scratch paint or chip the windshield.
Humans can't intuit mass unless the object meets certain criteria, either. What if it's an opaque cardboard box in the road?
> anything new It is nothing new. It is called (at least) "the problem of transparency". The chess context is probably not the best, because many systems allow a lengthy complex explanation of the response (I cannot remember now the exact workings of Deep Blue - it has been a while last time I met the full info). It is a real problem in general, because we may not just want responses but we may want " to learn someth…
> And of course, "understanding" the engine is required to advance it. I don't think so. People are clearly making huge advances in machine learning even though they can't explain the systems made 10 years ago, let alone more recent ones. There's a lot of trial and error / cargo-culting involved. Some researchers go as far as comparing machine learning research to Alchemy: https://www.science.org/content/article/ai-r…
But an AI must “understand its reasoning” in order to revise it efficiently to learn from its mistakes. If revision is impossible, the only way to correct its reasoning is to re-train the system from scratch. But in doing that, you can only hope the problem is fixed, and worse, that new problems were not just created.