Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is AI Riding a One-Trick Pony?

technologyreview.com

Post reply on HN
Print view View original

171–180 of 219 posts

Re: Is AI Riding a One-Trick Pony?

#171

Post by zby »

gadjo95 wrote: post #50

The most relevant part of the article: David Duvenaud, an assistant professor in the same department as Hinton at the University of Toronto, says deep learning has been somewhat like engineering before physics. “Someone writes a paper and says, ‘I made this bridge and it stood up!’ Another guy has a paper: ‘I made this bridge and it fell down—but then I added pillars, and then it stayed up.’ Then pillars are a hot ne…

But maybe there are no universal laws that govern AI like physics governs bridges? AI is something that finds universal laws in stuff - there is no meta level over this - all the meta is AI itself.

Re: Is AI Riding a One-Trick Pony?

#172

Post by allenz »

rawnlq wrote: post #146
karpathy wrote: post #132

There is a bit of "can't see the forest for the trees" failure in the article. AI is spearheading a paradigm shift in how we write programs. Or rather, we don't write programs. We write much much shorter programs that search the program space for programs that satisfy some desiderata. The programs we get as the output of the search process are extremely flexible, work very well, are very homogeneous in compute (e.g.…

Sounds like AI behaves similar to TDD where you blindly refactor until green, except AI have like a zillion more test cases(training data) to pass? Is that really a better way of writing code? (for example compared to being able to reason about the code to create something provably correct)

It's a trade-off. We lose explainability, but we are able to solve completely new classes of problems.

Re: Is AI Riding a One-Trick Pony?

#173

Post by edanm »

visarga wrote: post #142
johansch wrote: post #15

Earlier quoted context omitted.

Are you thinking Elon's concern is about what we have today? Duh. Today's AI methods are great at pattern recognition (weak AI). The real risk is obviously a strong, generic AI. We don't know how to to get there, but we kinda do know that we will, so now is the time to think about risks. I really don't get why this is controversial or hard to understand. (My most cynical thought is that it's just about people doing w…

To make an analogy, it's like starting the discussion on plane based terrorism and the need for TSA the next week after the Wright brothers had their first flight. If that happened, it would have stalled progress in aviation. Andrew Ng said he isn't worried about overpopulation on Mars, either. The real risk now is not AI taking over, it's humans using AI to abuse other humans. I trust the emerging AGI to do what's r…

Some terrorism might have indeed been prevented if some of the security measures had been put in place earlier. E.g. locked cabin doors. I could make a similar argument like yours, "that's like worrying about nuclear safety programs just because we started building nuclear weapons".

Not that I'm saying it's worth it, mind you. It's a tradeoff between money/time/progress vs. safety. It's totally valid to decide that it isn't worth the tradeoff to stifle the industry right now.

The main difference though between the potential dangers of AI, vs. terrorism, IMO, is that the potential danger is much much larger with AI (as it was with nuclear weapons).

Re: Is AI Riding a One-Trick Pony?

#174

Post by edanm »

I think they're vastly underestimating the amount of other things in the field of AI that have been happening. This article is kinda like saying "Turing invented computers in the '40s, and everything else we've done since then has been based on that insight". Well, yeah, that's not necessarily a bad thing.

Only in this case, that would be overstating, because Deep Learning, as impressive and hyped as it is, is still only one area of the field. It's true that most of the hype is around that, because it has given us breakthroughs in image/video/audio/text applications. But I'd still wager that most "AI" systems in the world use more traditional techniques, especially if you're looking at the myriad data scientists using things as simple as linear regressions.

And even within deep learning, there have been interesting advances, e.g. GANS have brought some very interesting applications (like style-transfer). Who knows, maybe in 30 years time people will be writing about how everything nowadays is built on GANS or deep reinforcement learning, a 30-year-old technique!

Re: Is AI Riding a One-Trick Pony?

#175

Post by nopinsight »

Animats wrote: post #155

I'd argue that the next problem to attack is manipulation in unstructured environments. Robots suck at that. There's been amazingly little progress in the last 40 years. DARPA had a manipulation project and the DARPA humanoid challenge a few years ago, and they got as far as key-in-lock and throwing a switch. Amazon is still trying to get general bin-picking to work. Nobody has fully automatic sewing that works well,…

I agree with your general characterization of the area. For sewing though, Sewbots appears production-ready and does not require starching the fabric. What do you think of it?

http://softwearautomation.com/products/

Re: Is AI Riding a One-Trick Pony?

#176

Post by FrozenVoid »

Instead of bigger and badder networks with zillions of layers, can we use the opposite approach: reducing the network to minimal size at which it works and researching why it works(Reduced case -> General rule). It will be much simpler to make current network easier to analyze, than trying to find relations in a giant multilayer soup. Perhaps even creating a neural network optimizer to minify the target network, increasing its efficiency without compromising the power at the tasks it built for.

Re: Is AI Riding a One-Trick Pony?

#177

Post by eutectic »

ridgeguy wrote: post #128
dragontamer wrote: post #87

Earlier quoted context omitted.

> 1) learn how the brain works 2) build a simulator I disagree that step #1 is important. Consider the "Air-foil", which led to flight. In one sense, its an approximation of the wings of birds and other animals. But ultimately, the discovery that the "Air-foil" shape turns sideways blowing wind into an upward force now called "lift" is completely different from how most people understand bird wings. Bird Wings flap,…

Disclaimer: I have no expertise in AI. That said, I agree that learning how the brain works seems unimportant and unnecessary. Evolution doesn't know how a brain works, but it's given us Einstein, Michelangelo, and conversations on HN. It seems really important to learn how to build evolution into attempts at AI, given that evolution is the only known mechanism that leads to what we recognize as intelligence.

You could have made exactly that this argument about flight before the Wright brothers.

Re: Is AI Riding a One-Trick Pony?

#178

Post by Retric »

bitL wrote: post #103
openasocket wrote: post #94

Earlier quoted context omitted.

I completely agree with your assessment, but the problem is a bit worse in my opinion. We already have a pretty firm grasp of how different ML systems learn and converge towards a solution in the average case. It's not that we need to understand our neural networks better, it's that we need to understand our problem domain better. We can't determine how well some ML architecture will perform at an object recognition…

The whole point of Deep Learning is that we don't want to describe math behind object recognition; it was the failed "classical" approach where people spent decades figuring out complex features which worked horribly. Deep Learning is actually pretty simple, well understood and parallelizable, and it's basically a billion-dimensional non-linear optimization. As optimization is infested with NP-hard problems, it's as…

Deep learning does not seem to abstract very well. Train on a data set then test with images that are simply upside down and the preformance can be significant.

Feature extraction also works much better when you toss a lot of data and processing power behind it. So, a lot of progress is simply more data and computing power vs better approaches. Consider how poorly deep leaning works when using a single 286.

Re: Is AI Riding a One-Trick Pony?

#179

Post by eternalban »

joe_the_user wrote: post #109
openasocket wrote: post #108

Earlier quoted context omitted.

It will probably play out like a conversation. A data scientist trains an ML model, and in analyzing the results discovers some intrinsic property or invariant of the problem domain. The scientist can then encode that information into the model and retrain. And that goes on and on, each time providing more accurate results. As an aside, I think it's important that we find a way to examine and inspect how an ML model…

A data scientist trains an ML model, and in analyzing the results discovers some intrinsic property or invariant of the problem domain. The scientist can then encode that information into the model and retrain. And that goes on and on, each time providing more accurate results. Mmmaybe, It's tricky to articulate what pattern the data-scientist could see ... that an automated system couldn't see. Or otherwise, perhaps…

The human participant may consider multiple distinct machine results, each a point in the space of algorithm, data set, bias applied to the problem domain. Human intuition is injected into the process and the result will be greater than the sum of the machines and a lone human mind.

What is interesting to note, now that above idea is considered, is that this process model itself belongs to the set of human-machine coordinations. Another process model is where low level human mind is used to perform recognition tasks too hard (or too slow) for machine to perform, for example using porn surfers to perform computation tasks via e.g. captcha like puzzles.

Long term social ramifications of all this is also interesting to consider as it motivates machines to breed distinct types of humans ;)

Re: Is AI Riding a One-Trick Pony?

#180

Post by pinouchon »

karpathy wrote: post #132

There is a bit of "can't see the forest for the trees" failure in the article. AI is spearheading a paradigm shift in how we write programs. Or rather, we don't write programs. We write much much shorter programs that search the program space for programs that satisfy some desiderata. The programs we get as the output of the search process are extremely flexible, work very well, are very homogeneous in compute (e.g.…

Just a quick question/remark: I have a feeling that it's best to think about DNNs as approximating a function, not a program. (only then you obtain a program as a result of applying this function). But because mathematically, you can formalise your big NN as one big parameterised function, I think it's more correct to view a NN as a function...

Optimisation in program space would be trying to find both the structure (connections and activation functions) and weights of the NN, which is not what we do currently. We tend to hand/engineer (or keep what works best empirically) the architecture (1), then train by finding the best weights.

I am very interested in approaches to efficiently opmitise in program space, and DL/backprop doesn't feel like it

(1) although this is starting to change.

Post reply on HN