Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An understanding of AI’s limitations is starting to sink in

economist.com

Post reply on HN
Print view View original

341–350 of 403 posts

Re: An understanding of AI’s limitations is starting to sink in

#341

Post by p1esk »

wpietri wrote: post #322
p1esk wrote: post #255

Earlier quoted context omitted.

a process of internal generation of notions How is this "knowledge extraction from data" process different?

Different from what? Regular communication? Because the first involves trying to sync up two minds to have the same idea. The latter involves one mind, trying to generate an idea that ends up being useful. Useful in the George Box sense: "All models are wrong, some models are useful."

I meant how is knowledge extraction done by a language model different from knowledge extraction done by a human scientist?

Re: An understanding of AI’s limitations is starting to sink in

#342

Post by HarHarVeryFunny »

mattlondon wrote: post #53

The trouble is that people have been sold this idea that ML/AI can do amazing things, without properly being told that really the things it can do are quite narrowly-scoped. They've been sold the Star Trek computer idea. For example, years ago I was working on a prototype/proof-of-concept thing for instrumenting industrial machinery with stick-on small computers. Simple stuff - attach accelerators, temperature, humid…

The best way to think of today's AI capability is: automation.

You train it with a lots of examples of "given this input, this is the output i want", and hopefully it learns to get the "correct" (similar input => similar output) output for new inputs that you feed it. i.e. you've now automated the process of figuring the correct output for a given input.

There is also the "reinforcement learning" AI paradigm where the trained AI is choosing actions (from a given repertoire) in order to maximize an action outcome score based on some scoring function you provided. This is appropriate in a situation where you want the "AI" to do something more than just select the correct output, but again no magic - you're having to anticipate and score the potential outcomes.

Re: An understanding of AI’s limitations is starting to sink in

#343

Post by legolas2412 »

bufferoverflow wrote: post #217
joe_the_user wrote: post #169

Earlier quoted context omitted.

You're being a pedantic. Human beings are tremendously better at driving than machines despite sometimes saying hexagonal rather than octagonal. Humans and current AIs both make mistakes but humans manage a kind of robustness, ability to deal gracefully with unexpected situations, that current AIs don't seem to be progressing towards.

> Human beings are tremendously better at driving than machines Human drivers: 1 death per 88 million miles traveled (in the US) [1] Tesla Autopilot: 5 deaths per 3 billion miles [2] [1] https://www.iihs.org/topics/fatality-statistics/detail/state... [2] https://electrek.co/2020/04/22/tesla-autopilot-data-3-billio... and https://en.wikipedia.org/wiki/List_of_self-driving_car_fatal...

Yeah. Totally makes sense to compare a human driver driving an average 7 year old 30k$ car with not so good safety ratings driven by average person in snowy, rainy pothole ridden roads to a newish 70k$ luxury car with good airbags/crumple zones driven in mostly sunny California roads by mostly young drivers with a driver assistance system.

And then for you to argue that the driver assistance system is actually better than a human driver if given the car alone!

Wow

Re: An understanding of AI’s limitations is starting to sink in

#344

Post by HarHarVeryFunny »

a1369209993 wrote: post #316
mattlondon wrote: post #53

The trouble is that people have been sold this idea that ML/AI can do amazing things, without properly being told that really the things it can do are quite narrowly-scoped. They've been sold the Star Trek computer idea. For example, years ago I was working on a prototype/proof-of-concept thing for instrumenting industrial machinery with stick-on small computers. Simple stuff - attach accelerators, temperature, humid…

> the possible MNIST digits are 0-9 Except - and this rather ties into your point - those are not the only possible digits; your network also has to deal with (ie reject) other possible digits such as "P", "E", "3̸̶", or "[Forlorn Sigil of Amon-Gül redacted]"[0], which look like, but are not, decimal digits. 0: https://www.youtube.com/watch?v=ajGX7odA87k

That depends on how you train it.

If you only train it on examples of 0-9 then those are the only outputs it's going to give. If you fed a "P" into such a net then the outputs would be the degree of similarity of that "P" to each of the (0-9) digits it was trained on. You could of course threshold the output and ignore any prediction with confidence less than, e.g., 90%.

If you wanted the net to do a better job of rejecting non-digits, or at least some specific ones, then you could include a bunch of non-digit examples in your training data (so now your net has 11 outputs: 0-9 and "non-digit"), then hopefully - but not necessarily - it's highest confidence prediction will be "non-digit" when presented with a non-digit input.

Re: An understanding of AI’s limitations is starting to sink in

#345

Post by dumb1224 »

HarHarVeryFunny wrote: post #342
mattlondon wrote: post #53

The trouble is that people have been sold this idea that ML/AI can do amazing things, without properly being told that really the things it can do are quite narrowly-scoped. They've been sold the Star Trek computer idea. For example, years ago I was working on a prototype/proof-of-concept thing for instrumenting industrial machinery with stick-on small computers. Simple stuff - attach accelerators, temperature, humid…

The best way to think of today's AI capability is: automation. You train it with a lots of examples of "given this input, this is the output i want", and hopefully it learns to get the "correct" (similar input => similar output) output for new inputs that you feed it. i.e. you've now automated the process of figuring the correct output for a given input. There is also the "reinforcement learning" AI paradigm where th…

In my opinion this type of tasks is the typical AI tasks that get most exposed to the general public. The AI/ML model is set to attempt a human task. The aspiration is to get as good as a human (or faster/better more precision etc). Classic statistical inference/ models usually don't perform very well in these scenarios(or do they?).

The typical AI methods train a model that utilises features that don't make much sense to humans (pixels/texture/word tokens etc). In contrast with traditional statistical modelling where each feature were given 'meaning' and their importance explored via investigating the observations (data) using properties of well studied mathematical models, the AI methods that often made the press don't put a lot of emphasis on these properties. The advancement is a push to utilise all available information/data to outstanding 'performance'. The explanation of the inner features within that math space is usually secondary (though I don't mean that authors of novel methods don't care about mathematical modelling).

I might be too naive here but that's how I feel after trying out many methodologies in my field.

Re: An understanding of AI’s limitations is starting to sink in

#346

Post by robbrown451 »

jpttsn wrote: post #283
robbrown451 wrote: post #99

Earlier quoted context omitted.

Whether 50s sci-fi imagined or implied other things is irrelevant to the question of whether or not the current capability described (point camera at sign, get translation) qualifies as AI. The point is we have current things that are quite amazing, and would at one time have been considered to be the sort of thing that only an AI would be able to do, and yet we keep moving the goalposts. As if AI is defined as "that…

But was it the “capability” that qualified as AI in the 50s? Or was the capability just one example of what the AI could do? Suppose we said we’ve invented Jesus because we’ve invented ways to walk on water and turn it into wine.

I don't get your Jesus analogy, I mean Jesus is a proper noun for an individual.

A technology is different. Its capabilities pretty much define it. Unless you are going to try to get all philosophical about it and say it doesn't count if it doesn't experience qualia or something (which is nonsense) Or, unless you are defining it in ways that specifically call out the implementation details. A helium balloon isn't a hot air balloon, not because it doesn't have capabilities, but because you've specifically said in the name that it must use hot air.

Re: An understanding of AI’s limitations is starting to sink in

#347

Post by HarHarVeryFunny »

01100011 wrote: post #149
mattlondon wrote: post #53

The trouble is that people have been sold this idea that ML/AI can do amazing things, without properly being told that really the things it can do are quite narrowly-scoped. They've been sold the Star Trek computer idea. For example, years ago I was working on a prototype/proof-of-concept thing for instrumenting industrial machinery with stick-on small computers. Simple stuff - attach accelerators, temperature, humid…

> currently ML/AI requires us to know what the possible answers can be before we even begin training the network Isn't that what unsupervised learning is for?

There's no such thing as truly unsupervised learning - any machine learning algorithm is defined by some guidelines you've provided (or built-in) for what output it has to generate.

For example, you might think of data clustering as an unsupervised problem, but in reality you're still providing supervision in terms of what similarity measure to use and some control over how many clusters it should generate; and, at the end of the day, the output is always going to be a set of clusters, not, say, a stock tip or wry comment on the nature of your data!

Re: An understanding of AI’s limitations is starting to sink in

#348

Post by sevensor »

TrackerFF wrote: post #278
mattlondon wrote: post #53

The trouble is that people have been sold this idea that ML/AI can do amazing things, without properly being told that really the things it can do are quite narrowly-scoped. They've been sold the Star Trek computer idea. For example, years ago I was working on a prototype/proof-of-concept thing for instrumenting industrial machinery with stick-on small computers. Simple stuff - attach accelerators, temperature, humid…

That's actually something ML is incredibly useful at, when it comes to machines with sensors - failure prediction / anomaly detection, etc. In the industry, (preventive) maintenance takes up a pretty huge chunk of resources. It's something techs need to do often, and it's often a laborious task, but it's obviously done to reduce downtime. So the business insight, as they like to call it, is to reduce costs tied up to…

I work in this space, and you've called out the thing that drives me absolutely bonkers. No matter how accurately I can diagnose the state of the equipment, what everybody wants is to know how long until it explodes, so that they can take it down for maintenance ten minutes before that.

Re: An understanding of AI’s limitations is starting to sink in

#349

Post by Balgair »

mattlondon wrote: post #53

The trouble is that people have been sold this idea that ML/AI can do amazing things, without properly being told that really the things it can do are quite narrowly-scoped. They've been sold the Star Trek computer idea. For example, years ago I was working on a prototype/proof-of-concept thing for instrumenting industrial machinery with stick-on small computers. Simple stuff - attach accelerators, temperature, humid…

So, uh, how do you talk to the C_Os about this and tell them that it's GIGO?

Asking for a friend.

Re: An understanding of AI’s limitations is starting to sink in

#350

Post by smabie »

bcrosby95 wrote: post #55
bsaul wrote: post #48

Earlier quoted context omitted.

since you’re working on the field : i’m still under the impression that modern ML (neural network in particular) are not producing any science, nor are they producing any level of understanding of the phenomenon it’s trying to model. And that as such, we can’t provide a reliable estimate of the limitations of the produced model, other than by feeding it tons of inputs and measure the result. It would be like trying t…

Some of this would be considered a feature by companies. It's more defensible to have unknowable AI deciding to do illegal things than programmers hard coding illegal things. Which really boggles my mind. When my kid does something illegal I'm held liable. When an ML algorithm programmed by a team of people does, nothing we can do about that!

>When my kid does something illegal I'm held liable

This isn't true.

Post reply on HN