"The result is an artificial idiot savant that can excel at well-bounded tasks, but can get things very wrong if faced with unexpected input." I think this gets to the core of what is still a limitation of current technologies. Venturing into the unknown is still a deeply relevant task that seems unlikely to be replaced by computers anytime soon.
This is changing. In natural language processing just in the past couple weeks OpenAI wrote about their GPT3 model which can learn some tasks remarkably quickly, after only 1 or 2 examples. That model has extreme compute requirements, but it shows strong progress on performing never before seen tasks. There is still some steam left in the current deep learning boom. https://arxiv.org/abs/2005.14165
An understanding of AI’s limitations is starting to sink in
291–300 of 403 posts
Re: An understanding of AI’s limitations is starting to sink in
#292It's limited but so effective! The other day a friend asked for photos of her sister (also a friend) that I had because it's her birthday and she wanted to make a collage. I just searched on Google Photos by her name and it found a bunch because of the face classification. That's some good shit.
Re: An understanding of AI’s limitations is starting to sink in
#293The trouble is that people have been sold this idea that ML/AI can do amazing things, without properly being told that really the things it can do are quite narrowly-scoped. They've been sold the Star Trek computer idea. For example, years ago I was working on a prototype/proof-of-concept thing for instrumenting industrial machinery with stick-on small computers. Simple stuff - attach accelerators, temperature, humid…
That's actually something ML is incredibly useful at, when it comes to machines with sensors - failure prediction / anomaly detection, etc. In the industry, (preventive) maintenance takes up a pretty huge chunk of resources. It's something techs need to do often, and it's often a laborious task, but it's obviously done to reduce downtime. So the business insight, as they like to call it, is to reduce costs tied up to…
That's not what they wanted.
What people are being sold is AI/ML as a magic bullet that will do something useful regardless of the situation, and it lets business people avoid making decisions about what they actually want, because AI/ML can be anything, so they just signup for it and expect to get 20 things they didn't know they wanted handed to them on a plate.
Turn out, it's not enough to just collect a bunch of data and wave your magic wand at it. It wasn't with web analytics 10 years ago, it's still not.
What you actually need is someone who has a bunch of tricks up their sleeve, and has done this before, and can suggest a bunch of Business Insights the business might need before they start building anything, people that actually decide what to do, and actions taken to investigate, and solve those problems.
I mean, to some degree you're right; perhaps ML models could be useful for tracking hardware failures, but that's not what the parent post is talking about. The previous post was talking about just collecting the data and expecting the predictive failure models to just jump out magically.
That doesn't happen; it needs a person to have the insight that the data could be used for such a thing, and that needs to happen before you go and randomly collect all the wrong frigging metrics.
...but hiring experts is expensive, and making decisions is hard. So ML/AI is sold like snake-oil to managers who want to avoid both of those things. :)
Re: An understanding of AI’s limitations is starting to sink in
#294Earlier quoted context omitted.
That's actually something ML is incredibly useful at, when it comes to machines with sensors - failure prediction / anomaly detection, etc. In the industry, (preventive) maintenance takes up a pretty huge chunk of resources. It's something techs need to do often, and it's often a laborious task, but it's obviously done to reduce downtime. So the business insight, as they like to call it, is to reduce costs tied up to…
Yep, you and the user you're replying to are both right in different ways. One thing's for sure - machines don't generate "insights" on their own. Let's define an "insight" as "new meaningful knowledge", just for fun. We could talk about what comprises "new" and "meaningful" but it would be beside the point I'm making. In a supervised learning problem, the range of possible outputs is already known, meaning the model…
Yup, this is something I've seen from both sides. First you mention is basically the standard, while the last is part of the deep learning voodoo black magic that executives and sales love.
I've had people approach me with proposals like "What if we just churn [ALL OF] our data through this or that model, and let's see if it comes up with some patterns we've never seen or thought about"
And that's not just for industrial applications. It's everywhere.
What is concerning to me is that this mentality will surely induce more unrealistic expectations. Before you know it, business execs are starting to ask why we need business analysts at all, because surely those fancy deep neural networks can extract all kinds of features - "only need data scientists to figure out those things".
So yeah, that's my fear. That businesses will blindly start to discard domain knowledge, and just feed black-box models their data, and let the data scientists wrestle with the results.
Re: An understanding of AI’s limitations is starting to sink in
#295Earlier quoted context omitted.
Sure, ML is not at human-level intelligence - it is very much a tool-AI where we give a task to the machine and let it get very good at that. Nonetheless, it seems like progress on that front has been made incredibly quickly. Sure, Siri might not always "understand" what you're asking but the ML is able to very accurately transcribe your voice to text, something not really possible a few decades ago. Image classifier…
Progress in speech recognition has been slow and incremental, not fast and impressive. Hidden Markov Models did a decent job in the 90s. Now we have more data and more computing power.
We have seen huge progress in a number of other fields as well. Anecdotally, voice recognition has definitely gotten way better as well.
Re: An understanding of AI’s limitations is starting to sink in
#296Earlier quoted context omitted.
Absolutely. To be perfectly honest, it surprises me the extent to which ML naysaying seems to be popular on HN. The evidence of enormous progress seems pretty obvious to me.
> To be perfectly honest, it surprises me the extent to which ML naysaying seems to be popular on HN. Maybe because there is a larger fraction of people who know or see how the sausages are made. There are cool things, but is not magical nor transformative. At least not yet.
Re: An understanding of AI’s limitations is starting to sink in
#297I'm not sure how anyone who's watched the exponential growth of a brand new domain can pick a point today to and say that things aren't as good as we expected. What may have happened was that some eager CEOs have overpromised on timelines and resources. But the revolution is coming, ML is already starting to change society. We're building the tech. Right now. The author does not even realise the immeasurable potentia…
> There's far more to do with ML and AI than self driving cars and shitty ad recommendations. Yeah, there's also shitty sentencing recommendations[1], new-age phrenology[2], and high-tech redlining[3]. I think your entire field needs to take a year off and take some ethics and philosophy courses before going any further. Otherwise we're all going to end up much worse off. [1] https://www.nytimes.com/2017/10/26/opinio…
Should we have?
Re: An understanding of AI’s limitations is starting to sink in
#298Earlier quoted context omitted.
I completely agree with this. Let me add a non-US perspective that may surprise some people. I think a big part of what is holding back many companies from making effective, genuine, real-world use of AI is that a significant majority of the individuals involved are bad at their jobs . On the business side, there is an widespread unwillingness to acknowledge that technical people may be better placed to make decision…
Your argument boils down to “AI won’t be useful unless humans are twice as smart as they are” (your examples of the businesspeople and researchers), and thus doesn’t really say anything.
Re: An understanding of AI’s limitations is starting to sink in
#299I'm not sure how anyone who's watched the exponential growth of a brand new domain can pick a point today to and say that things aren't as good as we expected. What may have happened was that some eager CEOs have overpromised on timelines and resources. But the revolution is coming, ML is already starting to change society. We're building the tech. Right now. The author does not even realise the immeasurable potentia…
Re: An understanding of AI’s limitations is starting to sink in
#300Earlier quoted context omitted.
There has been next to ZERO progress towards genuine AGI despite a never-ending deluge of AI articles; that's normally the cause of scepticism. After several decades and a much-hyped last few years we have fake cleverness - impressively so in both cases - but nothing more.
Disagree. GPT-2/GPT-3 are able to pass for humans when the reader isn't paying close attention. This article seems insightful to me: http://www.overcomingbias.com/2017/03/better-babblers.html A scarily large amount of human speech is essentially word prediction, especially in cases where someone wants to seem impressive without having actually done the work. We're all familiar with the problem of people "bullshitting…
Tic Tac Toe, Chess, Image Classifying, Translation, Go, Sentence Construction, etc.; We are creating phenomenally impressive calculating worm-equivalents, but nothing human-level, or even remotely so.