Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Is No Match for the Quirks of Human Intelligence

thereader.mitpress.mit.edu

Post reply on HN
Print view View original

51–60 of 86 posts

Re: AI Is No Match for the Quirks of Human Intelligence

#51

Post by SavantIdiot »

bingohbangoh wrote: post #34

At what point does the current AI hype machine crash? It feels like these startups all sell solutions to the top 20 or so companies, competing for a very limited market. It's table stakes for everybody else, an "all-or-nothing" achievement for AI. By contrast, even a small amount of connectivity improved things dramatically for huge swaths of industry. That's how the internet started small and grew up. AI seems to re…

I can't speak to cloud deep learning, but edge NN's have two practical applications: anomaly detection and object defect detection. (Keyword spotting is completely stalled. I've been watching Arm-based companies work on keyword spotting for years and they are still 80% accurate in a real setting, which is like talking to an old person with hearing issues.) Anomaly detection (vibration analysis, audio anomalies) are working fairly well. Same goes for assembly lines where defective parts are booted off, and NN's don't improve that much over traditional methods.

The hype machine is all the big sexy stuff. GPT, Protein folding and Alpha Go are truly amazing. But I haven't seen a big step-function in deployable NNets (github-ready) in years. Which given the number of people researching this I expected a linear progression. What have we seen since UNet, MobileNet, SSD, Yolo, BERT, ResNet150, Inception ... ? That's worrisome, but it also could be the pendulum swing while people figure out exactly why these NN's work the way they do.

I think we need to wait another 20 years, which means lots of hype money will dry up, but I don't think it will vanish. The efficiency of neural nets as a general function approximation might not be super high accuracy, but it is amazing to see such a simple construct perform with such speed.

Re: AI Is No Match for the Quirks of Human Intelligence

#52

Post by bingohbangoh »

SavantIdiot wrote: post #51
bingohbangoh wrote: post #34

At what point does the current AI hype machine crash? It feels like these startups all sell solutions to the top 20 or so companies, competing for a very limited market. It's table stakes for everybody else, an "all-or-nothing" achievement for AI. By contrast, even a small amount of connectivity improved things dramatically for huge swaths of industry. That's how the internet started small and grew up. AI seems to re…

I can't speak to cloud deep learning, but edge NN's have two practical applications: anomaly detection and object defect detection. (Keyword spotting is completely stalled. I've been watching Arm-based companies work on keyword spotting for years and they are still 80% accurate in a real setting, which is like talking to an old person with hearing issues.) Anomaly detection (vibration analysis, audio anomalies) are w…

> edge NN's have two practical applications: anomaly detection and object defect detection

Do you have any links to companies working on this?

I also haven't found any real innovations since adversarial neural networks in ~2016, which roughly fits in with your timeline. Not sure what happened to Geoff Hinton's Capsule networks, that looked like it was going to be a big thing at the time. This is admittedly a very hot take though.

Re: AI Is No Match for the Quirks of Human Intelligence

#53

Post by belval »

bingohbangoh wrote: post #34

At what point does the current AI hype machine crash? It feels like these startups all sell solutions to the top 20 or so companies, competing for a very limited market. It's table stakes for everybody else, an "all-or-nothing" achievement for AI. By contrast, even a small amount of connectivity improved things dramatically for huge swaths of industry. That's how the internet started small and grew up. AI seems to re…

It won't because actual DL products solve problems that are not sexy but still works objectively better than traditional ML for most tasks.

The best example of this is OCR/Document Intelligence. The old ML approaches of the 1990-2000 are really no match for what we have now. It's gotten to a point where extracting handwritten text is done reliably for very cheap. Same goes for tables and key-values extractions which require a higher level of understanding that simply wasn't available before.

So no, it won't crash and you'll keep seeing these pieces every month that "AI can't do X" until we actually achieve X and then the goalpost is moved.

Re: AI Is No Match for the Quirks of Human Intelligence

#54

Post by paul7986 »

esalman wrote: post #25
paul7986 wrote: post #7

If true then robot cars are scarier then I've thought ... AI is going to be able to process the right scenario to avoid a deadly crash within the first decade or more of market adoption or the programmers are going to learn and ship bug fixes for each person their prior code killed? *I'm always downvoted for this thought but is this not the true way forward with robot cars? Those who downvote me think the first batch…

Well, autonomous cars are able to detect obstacles on an narrow lane. That is nowhere near general intelligence.

I wonder how they do on the L.A. freeway with average traffic that goes to heavy and average again where there's a police chase happening in a torrential downpour... im sure more scenarios that happen in life at once can be added on ... yet don't worry the AI knows all the millions of scenarios on top each other and will be able to handle it like an attentive human driver would in such situations.

Re: AI Is No Match for the Quirks of Human Intelligence

#55

Post by SavantIdiot »

bingohbangoh wrote: post #52
SavantIdiot wrote: post #51

Earlier quoted context omitted.

I can't speak to cloud deep learning, but edge NN's have two practical applications: anomaly detection and object defect detection. (Keyword spotting is completely stalled. I've been watching Arm-based companies work on keyword spotting for years and they are still 80% accurate in a real setting, which is like talking to an old person with hearing issues.) Anomaly detection (vibration analysis, audio anomalies) are w…

> edge NN's have two practical applications: anomaly detection and object defect detection Do you have any links to companies working on this? I also haven't found any real innovations since adversarial neural networks in ~2016, which roughly fits in with your timeline. Not sure what happened to Geoff Hinton's Capsule networks, that looked like it was going to be a big thing at the time. This is admittedly a very hot…

At Embedded World 2020, every embedded semi and FPGA company had some kind of NNet running. One that was really impressive was Xilinx. They had multiple demos set up at Embedded World in 2020 showing NNet's for defects, person detection, and audio vibrational analysis with autoencoders. As did several other FPGA companies. I googled some links but they don't look like what I saw. So I'd recommend googling Xilinx if you are interested in industrial applications.

Re: AI Is No Match for the Quirks of Human Intelligence

#56

Post by bingohbangoh »

SavantIdiot wrote: post #55
bingohbangoh wrote: post #52

Earlier quoted context omitted.

> edge NN's have two practical applications: anomaly detection and object defect detection Do you have any links to companies working on this? I also haven't found any real innovations since adversarial neural networks in ~2016, which roughly fits in with your timeline. Not sure what happened to Geoff Hinton's Capsule networks, that looked like it was going to be a big thing at the time. This is admittedly a very hot…

At Embedded World 2020, every embedded semi and FPGA company had some kind of NNet running. One that was really impressive was Xilinx. They had multiple demos set up at Embedded World in 2020 showing NNet's for defects, person detection, and audio vibrational analysis with autoencoders. As did several other FPGA companies. I googled some links but they don't look like what I saw. So I'd recommend googling Xilinx if y…

many thanks!

Re: AI Is No Match for the Quirks of Human Intelligence

#57

Post by PartiallyTyped »

omegalulw wrote: post #50
PartiallyTyped wrote: post #42

Earlier quoted context omitted.

We have beat humans on every single atari game by at least one order of magnitude, and we do that consistently, and it really only took 5 or so years since the first solution that provided tangible results. It has only been 8 or 9 years since GPGPUs were used for ML research. We are also seeing models that are able to generate code given prompts. Given enough representational power, I don't see why a model that learn…

> We have beat humans on every single atari game by at least one order of magnitude There's mechanical skill involved, it's not purely intelligence. > We are also seeing models that are able to generate code given prompts. This has been discussed a lot, but the generated code is nowhere close to good enough for large projects where you really need intelligence. > Given enough representational power, I don't see why a…

These two go hand in hand

> This has been discussed a lot, but the generated code is nowhere close to good enough for large projects where you really need intelligence.

> Except that it's not linear scaling. The larger NLP models consume absurdly large resources, it's not straightforward to "get enough representational power"

When allowing maximizers to run wild, just like reinforcement learning, they will find hidden solutions, and when the model can provide an action in the form of a dense representation, it can also use code generation models with much more precision that we do because it can skip the encoding part.

> Also, most models fail to adapt to new tasks outside of their narrow training scope, that's a massive problem. Even if you make models large, you will find that getting data covering all edge cases is exponentially expensive.

We are still 6-7 years in. Deepmind's last paper on general agents has them generalizing to new tasks relatively easily. It's still not there, but we miles ahead than we were 5 years ago.

Re: AI Is No Match for the Quirks of Human Intelligence

#58

Post by bingohbangoh »

belval wrote: post #53
bingohbangoh wrote: post #34

At what point does the current AI hype machine crash? It feels like these startups all sell solutions to the top 20 or so companies, competing for a very limited market. It's table stakes for everybody else, an "all-or-nothing" achievement for AI. By contrast, even a small amount of connectivity improved things dramatically for huge swaths of industry. That's how the internet started small and grew up. AI seems to re…

It won't because actual DL products solve problems that are not sexy but still works objectively better than traditional ML for most tasks. The best example of this is OCR/Document Intelligence. The old ML approaches of the 1990-2000 are really no match for what we have now. It's gotten to a point where extracting handwritten text is done reliably for very cheap. Same goes for tables and key-values extractions which…

> ... "AI can't do X" until we actually achieve X and then the goalpost is moved.

This gets said often but I don't think anybody whose credible in the field actually makes statements like this.

Take Chess for instance. In the IBM "Big Blue" documentary from ~2004, they quote journalists as saying "AI can't play Chess as well as humans but if it could, then AI would be solved." Why did the techniques from Big Blue not seem to go anywhere?

I know for a fact that scientists _were not saying this_. The Lighthill Debates on AI specifically talk about why playing games well doesn't really prove anything. [0]

I do agree that OCR has improved greatly by AI but this feels very niche. Somebody mentioned defect anomaly detection in another comment and I was not aware of this. All useful for sure. Still, this doesn't amount to anywhere near the hype that was announced earlier last decade. Moreover, the economics in AI are mostly awful despite everybody's seemingly best efforts. [1]

Even if its useful in some vague sense, it's not necessarily economically useful. Amazon has ~10,000 people working on Alexa. [2] Have they turned a profit on these endeavours? I understand they can absorb the costs but its not clear to me how the economics will work out here.

ML models haven't even been useful in places where statistical methods have reined supreme such as Renaissance Technologies and other hedge funds. No large companies are using neural networks in a significant capacity to my knowledge.

Another big tell for me is the lack of any consumer products in the space. Where did they go? why are they missing? This is what I mean by "everybody is competing for the top 20 or so customers."

This is compounded by the unstructured nature of most data. Most databases are still terrible, especially at the few institutions large enough to have it and large enough for it to make a difference in their business. There should be more focus into this problem if anything. A well tuned and structured database will be many times more useful than a fancy model that needs constant retraining. But I guess its not as cool so nobody cares.

[0]: https://www.youtube.com/watch?v=03p2CADwGF8 -- highly recommended, with many of the arguments still resonating today.

[1]: https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-i...

[2]: https://qr.ae/pGJUKk -- couldn't find a better source offhand.

Re: AI Is No Match for the Quirks of Human Intelligence

#59

Post by bdamm »

paul7986 wrote: post #48
comeonseriously wrote: post #44

Earlier quoted context omitted.

> and ship bug fixes for each person their prior code killed My question is always, who is responsible for those deaths? Or will we just have so little care for life that we consider them "for the cause"?

That's my sentiment on this .... progress will be a killer in this realm of software development and on a large scale. Developers who get into robot cars the deaths their AI makes doesn't or won't bother them? Just analyze the data, learn and ship the fix who cares about johnny, susie and their kid killed by their AI's mistake in which they weren't even driving a robot car just driving alongside one like Uber's pedes…

Paul, some engineers already live in the world where mistakes can cost lives. Including automotive software engineers. In fact, the engineering world is full of people who make compromises knowing that the product could be safer, but they're aiming for safe enough because that's the only real way to move forward.

Re: AI Is No Match for the Quirks of Human Intelligence

#60

Post by betenoire »

yk wrote: post #39

There's an entire genre of articles which argue that computers can never do X, because X is ill defined. Thing is, half of the time X is only ill defined because the writer neglects to define X well, and for the other half, that the writer can't define X well is no proof that it is impossible to define X well. Here however, if the take the argument of representations in a technical sense, then since a neural network…

I don't disagree necessarily, but humans seems capable of solving problems even when when they are poorly defined by understanding the some implicit context. Sure if we encode all the context, the computer can solve it too, but isn't that fundamentally what the article is trying to convey? That we haven't really taught computers to read between the lines yet?
Post reply on HN