Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is machine learning 'hard'? (2016)

ai.stanford.edu

Post reply on HN
Print view View original

71–80 of 144 posts

Re: Why is machine learning 'hard'? (2016)

#71

Post by sanroot99 »

galaxyLogic wrote: post #70
tbrownaw wrote: post #59

Earlier quoted context omitted.

> you don't often know if your data or model selection can produce the results you want. Like, not knowing if your data set actually contains anything predictive of what you're trying to predict?

The Farmer is Turkey's best friend. Turkey believes so because every day Farmer gives Turkey food, lots of food. Farmer also keeps the Turkey warm and safe from predators. Turkey predicts that the Farmer will keep on being his best friend also tomorrow. Until one day, around Thanksgiving, the prediction goes wrong, awfully wrong.

Three body problem, reference,yeh

Re: Why is machine learning 'hard'? (2016)

#72

Post by jxramos »

owlninja wrote: post #54
4death4 wrote: post #51

I used to work on an ML research team. In addition to what the author mentions, there is an entirely separate issue: whether or not what you're attempting to do is possible with the approach you've chosen. Consider making an iOS app. For the most part, an experienced software engineer can tell you if making a given app is possible, and they'll have a relatively clear idea about the steps required to realize the idea.…

You nailed where I currently stand. At my company I've been a jack of all trades but mostly software/dba work. My boss and I were very excited about ML when the hype cycle was taking off several years ago and completed a successful project. Fast forward to today, I got loaned out to another team that lost their data scientist, and for the first time in my career I'm having to say - "I don't think we can do what you w…

It can be a very empirical art. If you can't generate more data at the time you can sometimes invest in reviewing the hand labeling ground truth to verify no false classifications slipped by.

Re: Why is machine learning 'hard'? (2016)

#73

Post by dylan604 »

4death4 wrote: post #51

I used to work on an ML research team. In addition to what the author mentions, there is an entirely separate issue: whether or not what you're attempting to do is possible with the approach you've chosen. Consider making an iOS app. For the most part, an experienced software engineer can tell you if making a given app is possible, and they'll have a relatively clear idea about the steps required to realize the idea.…

I have the same sentiments about DIY electronic designs. If I take someone else's designs and build it at home, I know it's all on my build skills lacking if it doesn't work as there is already working examples. If I design a device from the electronics to the software, I don't know if the thing isn't working because of bugs in the code, problems with the build of the electronics, or fundamental flaw in the design itself. At least not without a ton of time debugging it all.

Re: Why is machine learning 'hard'? (2016)

#74

Post by dylan604 »

raincole wrote: post #56
4death4 wrote: post #51

I used to work on an ML research team. In addition to what the author mentions, there is an entirely separate issue: whether or not what you're attempting to do is possible with the approach you've chosen. Consider making an iOS app. For the most part, an experienced software engineer can tell you if making a given app is possible, and they'll have a relatively clear idea about the steps required to realize the idea.…

But aren't all science basically like this? If you know your hypothsis works before you do the experiments, it's not science anymore.

Sometimes something you've done works, but you really don't know why/how. You then have to walk it back to figure out by experimenting what is causing it to work. I feel like this happen(ed|s) in chemistry a lot. Was it the fact that I stirred it counter clockwise this time, or that I got distracted and the temp went 5° hotter than intended, or that I didn't quite clean my beaker properly and some residue contaminated this batch, or any number of other steps.

Re: Why is machine learning 'hard'? (2016)

#75

Post by chefandy »

wegfawefgawefg wrote: post #61
Cacti wrote: post #42

Earlier quoted context omitted.

You let newcomers dick around for months with a net on an industrial vision problem? This stuff was solved two decades ago. Why didn’t you just tell them?

Sometimes its really hard to convince an organization that their strategy wont work. Especially if ego or higher ups are convinced.

Many developers also mistakenly think their mighty dev super brain genius powers make them capable of accurately evaluating and critiquing any profession. I’ve had many developers try to explain design to me, even knowing I’m an experienced, educated professional designer. I can see why some might roll their eyes.

Re: Why is machine learning 'hard'? (2016)

#76

Post by Animats »

dylan604 wrote: post #73
4death4 wrote: post #51

I used to work on an ML research team. In addition to what the author mentions, there is an entirely separate issue: whether or not what you're attempting to do is possible with the approach you've chosen. Consider making an iOS app. For the most part, an experienced software engineer can tell you if making a given app is possible, and they'll have a relatively clear idea about the steps required to realize the idea.…

I have the same sentiments about DIY electronic designs. If I take someone else's designs and build it at home, I know it's all on my build skills lacking if it doesn't work as there is already working examples. If I design a device from the electronics to the software, I don't know if the thing isn't working because of bugs in the code, problems with the build of the electronics, or fundamental flaw in the design it…

However, we now have techniques for debugging electronics. Electronics tends to be designed to be decomposable into subunits, with some way to do unit testing. At least in the prototype, before it's shrunk for production. Test gear can be expensive, but it exists, all the way down to the wafer if needed.

That wasn't always the case. Electronics problems used to be more mysterious. The conquest of electronic design and debugging is what allows making really complex electronics that works. It really is amazing that smartphones work at all, with all those radios in that little case. That RF engineers can get a GPS receiver and a GSM transmitter to work a few centimeters apart is just amazing.

Machine learning isn't that far along yet. When it doesn't work, the tools for figuring out why are inadequate. It's not even clear yet if this is a technique problem which can be fixed with tooling, or an inherent problem with having a huge matrix of weights.

Re: Why is machine learning 'hard'? (2016)

#77

Post by hooande »

Debugging is a problem. But the real problem I'm seeing is our expectations as software developers. We're used to being able to fix any problem that we see. If a div is misaligned or a column of numbers is wrong we can open the file, find the offending lines of code and FIX it.

Machine learning is different because every implementation has a known error rate. If your application has a measured 80% accuracy then 20% of cases WILL have an error. You don't know which 20% and you don't get to choose. There's no way to notice a problem and immediately fix it, like you can with almost every other kind of engineering. At best you can expand your dataset, incorporate new models, fix actual bugs in the code. Doing those things could increase the accuracy up to, say, 85%. This means there will be fewer errors overall, but the one that you happened to notice may or may not still be there. There's no way to directly intervene.

I see a lot of people who are new to the field struggle with this. There are many ways to improve models and handle edge cases. But not being able to fix a problem that's in front of you takes some getting used to.

Re: Why is machine learning 'hard'? (2016)

#78

Post by p1esk »

janalsncm wrote: post #62
tbrownaw wrote: post #59

Earlier quoted context omitted.

> you don't often know if your data or model selection can produce the results you want. Like, not knowing if your data set actually contains anything predictive of what you're trying to predict?

Here’s an example of something similar. Say you have a baseline model with an AUC of 0.8. There’s a cool feature you’d like to add. After a week or two of software engineering to add it, you get it into your pipeline. AUC doesn’t budge. Is it because you added it in the wrong place? Is the feature too noisy? Is it because the feature is just a function of your existing features? Is it because your model isn’t big eno…

All of these hypotheses will take on the order of days to check.

OK, but you can check them, right? How is that different from a regular software bug?

Re: Why is machine learning 'hard'? (2016)

#79

Post by mikrotikker »

Animats wrote: post #76
dylan604 wrote: post #73

Earlier quoted context omitted.

I have the same sentiments about DIY electronic designs. If I take someone else's designs and build it at home, I know it's all on my build skills lacking if it doesn't work as there is already working examples. If I design a device from the electronics to the software, I don't know if the thing isn't working because of bugs in the code, problems with the build of the electronics, or fundamental flaw in the design it…

However, we now have techniques for debugging electronics. Electronics tends to be designed to be decomposable into subunits, with some way to do unit testing. At least in the prototype, before it's shrunk for production. Test gear can be expensive, but it exists, all the way down to the wafer if needed. That wasn't always the case. Electronics problems used to be more mysterious. The conquest of electronic design an…

I never understood the black magic behind things like 4g until I saw a teardown of some pole equipment and saw the the solid copper beam forming cavities inside. Blew my mind.

Re: Why is machine learning 'hard'? (2016)

#80

Post by dchichkov »

4death4 wrote: post #51

I used to work on an ML research team. In addition to what the author mentions, there is an entirely separate issue: whether or not what you're attempting to do is possible with the approach you've chosen. Consider making an iOS app. For the most part, an experienced software engineer can tell you if making a given app is possible, and they'll have a relatively clear idea about the steps required to realize the idea.…

On top of that, vast majority of engineers and researchers who had joined the field, only did it in the last few years.

While, like with many other fields, it takes decades to get to a level of a well-rounded expert. One paper a day, one or two projects a year. It just takes time. No matter how brilliant or talented you are.

And then the research moves on. And more is different. A GFLOPS shift to TFLOPS and then PFLOPS over a single decade is a seismic shift.

Post reply on HN