Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is machine learning 'hard'? (2016)

ai.stanford.edu

Post reply on HN
Print view View original

141–144 of 144 posts

Re: Why is machine learning 'hard'? (2016)

#141

Post by eru »

hgomersall wrote: post #125
immibis wrote: post #123

Earlier quoted context omitted.

Let's rephrase: If you have to solve a problem, you'd better hope that problem is an engineering problem rather than a science problem.

Even many engineering problems are too difficult to know a priori that they will work. No new knowledge is needed as such, just the bounds on what is possible might be fuzzy.

Yes, of course.

If you care about solving your specific problem, you want to avoid having to acquire new knowledge. That's risky. (And I don't just mean knowledge that's new to you, that you get from reading a book. I mean knowledge that's new to humanity as far as you can tell.)

Eventually, someone will have to acquire some new knowledge to drive humanity forward. Just like in war, someone will have to go and fight the opponent; but you still prefer to achieve your objectives without having to fight whenever possible.

Re: Why is machine learning 'hard'? (2016)

#142

Post by auntienomen »

sanroot99 wrote: post #71
galaxyLogic wrote: post #70

Earlier quoted context omitted.

The Farmer is Turkey's best friend. Turkey believes so because every day Farmer gives Turkey food, lots of food. Farmer also keeps the Turkey warm and safe from predators. Turkey predicts that the Farmer will keep on being his best friend also tomorrow. Until one day, around Thanksgiving, the prediction goes wrong, awfully wrong.

Three body problem, reference,yeh

A bit older than that. This joke predates Bell labs.

Re: Why is machine learning 'hard'? (2016)

#143

Post by dchichkov »

immibis wrote: post #121
DeathArrow wrote: post #104

Earlier quoted context omitted.

>AUC doesn’t budge. Is it because you added it in the wrong place? Is the feature too noisy? Is it because the feature is just a function of your existing features? Is it because your model isn’t big enough to learn the new feature? Is there a logical bug in your implementation? Or is it because lack of expertise and experience and because someone tries stuff blindly without understanding a bit in the hope they will…

Isn't that all of AI? I get the impression that not even the "experts" really understand what new techniques will get good results - they're guided by past successes, and have loose ideas about why past successes were successful, but can't really predict what else will work. It seemed like the tremendous success of transformer architectures was a surprise to everyone, who had previously been throwing stuff at the wal…

No, it was not a surprise. Transformers architecture resulted from systematic exploration at scale with Seq2Seq. And it was quite clear when this architecture came out that it was very promising.

The issue was not technology, it was lack of investment. In 2017 with a giant sucking sound, Autonomous Vehicles research took all the investment money and nearly all talent. Myself is a good example, I was working on training code generation models for my startup Articoder, using around 8TB of code, scrapped from GitHub. Had some early successes, automatic pull requests generated and accepted by human users, got past YC application stage into the interview. The amount of VC funding for that was exactly zero. I've filed a patent, put everything on hold and went to work on AVs.

As to watching things not stick for multiple decades, you simply had too few people working on this. And no general availability of compute. It was a few tiny labs, with a few grad students and little to no compute available. Very few people had a supercomputer in their hands. In 2010, for example, a GPU rig like 2xGTX 470 (that could yield some 2 TFLOP of performance) was an exception. And in the same year, the top conference, like NeuralIPS had attendance of around 600.

Post reply on HN