Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is machine learning 'hard'? (2016)

ai.stanford.edu

Post reply on HN
Print view View original

41–50 of 144 posts

Re: Why is machine learning 'hard'? (2016)

#41

Post by aleph_minus_one »

MichaelRo wrote: post #27
ramesh31 wrote: post #11

Earlier quoted context omitted.

Would there be enough of a financial incentive to do so? Seems like a prime startup opportunity.

>> Seems like a prime startup opportunity. Sometimes it's just ... hard. Apply some thought maybe before blindly parroting "profit!" Reporter: "Why is it hard to cure cancer?". Crowd: "Would there be enough of a financial incentive to do so? Seems like a prime startup opportunity!" Reporter: "Why is it hard to end World poverty?". Crowd: "Would there be enough of a financial incentive to do so? Seems like a prime sta…

> Reporter: "Why is it hard to cure cancer?". Crowd: "Would there be enough of a financial incentive to do so? Seems like a prime startup opportunity!"

What you want to optimize for is the money amount that you make at some quantile of the probablity distribution of the profits; say, the profits that are guaranteed in the best, say, 3 %, 5 %, 10 % or even 20 % of all possible outcomes. With a probablity of 97 % (if you choose the best 3 % of the outcomes), you won't make sufficient money if you attempt to cure cancer to be worth the risk, so the financial incentive is not there.

TLDR: Financial incentives do matter, but work differently from how many people think that they are structured.

Re: Why is machine learning 'hard'? (2016)

#42

Post by Cacti »

barelyauser wrote: post #33
sackfield wrote: post #31

I think a big difference between ML and regular programming is how the components at scale make the systems viable. When I was learning computer science, it seemed quite intuitive to me that you would start out with assembly, then go to a C like compiler, then abstract that to a JIT/dynamic type language, and go from that to the UI. I could see how each step in the layer added value and presented its tradeoffs. Contr…

When building models it is useful to spend some time finding out what you already know about the problem. Things you yet don't know you know. This kind of knowledge will greatly simplify the model. I see newcomers making this mistake very often. In industrial vision, for example, the newcomers like to create very complicated models. I then show them that the "box" you trained a entire model to recognize will actually…

You let newcomers dick around for months with a net on an industrial vision problem? This stuff was solved two decades ago. Why didn’t you just tell them?

Re: Why is machine learning 'hard'? (2016)

#43

Post by barelyauser »

Cacti wrote: post #42
barelyauser wrote: post #33

Earlier quoted context omitted.

When building models it is useful to spend some time finding out what you already know about the problem. Things you yet don't know you know. This kind of knowledge will greatly simplify the model. I see newcomers making this mistake very often. In industrial vision, for example, the newcomers like to create very complicated models. I then show them that the "box" you trained a entire model to recognize will actually…

You let newcomers dick around for months with a net on an industrial vision problem? This stuff was solved two decades ago. Why didn’t you just tell them?

They usually don't take months. Would be optimal if I could catch them at the get go. Not what happens most of the time. When that happen most of the times you will se a manager who is not technical leading a group selected by "professional" recruiters. There is a lot of waste out there.

Re: Why is machine learning 'hard'? (2016)

#44

Post by itissid »

Great teachers(if you can find them): Andrej Karpathy, Andrew Gelman & Ben Goodrich(Columbia), Subbarao Khambampati(ASU) to name a few I know of.

Go Where the hard problems are (or find someone who is doing it): If you don't have a good intuition where to get good problems to practice on for a pay, choose a place to work where a Data Scientist is not just building dashboards/analytics but the company/team relies on them for answer to questions like: "What goals should we set for the next half based on what you see"?

ML practitioner (read: use ML tech to do/debug X) different from ML Engineer (Read: Implement ML algorithm X e2e on data ) is different from Applied Statistician (think marketing sciences or powering experiments like A/B tests): All three areas of work in different areas of ML in one form or another. But make sure what you want to work in is clear in your head and your expectations from it.

A lot of ML/Stats can be not with big data and yet really intuitive: I would say look for a problem domain in social/life/pharma/eco/political/survey/edtech sciences. They are full of intuitive models that need to be explainable and are often debuggable. An example here is usage of Stan software for Multilevel/Heirarchical Regression problems. Training here also makes you a great DS.

Re: Why is machine learning 'hard'? (2016)

#45

Post by yshvrdhn »

I think one of the issues is that fixing a problem is a lot harder in ML than in software engineering. You know that the model fails on this particular data point. If you have identified a bug in the code and wrote a fix did a pull request as long you are able to test the code for conditions you failed on you would have solved the problem. With modern ml especially with nueral nets as long as you don't have a way to spin up a data engine to track the problems you are facing and collect similar points you problem is not fixed.

Re: Why is machine learning 'hard'? (2016)

#46

Post by Cacti »

barelyauser wrote: post #43
Cacti wrote: post #42

Earlier quoted context omitted.

You let newcomers dick around for months with a net on an industrial vision problem? This stuff was solved two decades ago. Why didn’t you just tell them?

They usually don't take months. Would be optimal if I could catch them at the get go. Not what happens most of the time. When that happen most of the times you will se a manager who is not technical leading a group selected by "professional" recruiters. There is a lot of waste out there.

Oh. I understand. Carry on then lol

Been there. And you’re right.

Re: Why is machine learning 'hard'? (2016)

#47

Post by robotresearcher »

ummonk wrote: post #19
epistasis wrote: post #4

Earlier quoted context omitted.

Love that thread. The top comment is excellent: > Like picking hyperparamters - time and time again I've asked experts/trainers/colleagues: "How do I know what type of model to use? How many layers? How many nodes per layer? Dropout or not?" etc etc And the answer is always along the lines of "just try a load of stuff and pick the one that works best". > To me, that feels weird and worrying. It's like we don't yet un…

> This embodies the very fundamental difference between science and engineering. Not really though. In engineering, you have heuristics, even if you don't know why they work. In the case of deep learning / AI, there seems to be very little in the way of built up heuristic knowledge - it's just "try stuff and see what works for every problem".

All the model topologies that have names are heuristics. The idea of a 'layer' is a heuristic. And so on.

You don't really just try stuff. You choose very few things to try from the space of models. And you choose craftily.

We have quite a lot of domain craftiness now, if you think about it that way.

Re: Why is machine learning 'hard'? (2016)

#48

Post by roenxi »

Machine learning isn't comparable to software development. It is a statistical modelling exercise. This is like asking why advertising is hard - if a non-expert wades in to a different domain then they may find it has different challenges than what they are used to! This is just a specific case of the normal things that analysts routinely deal with.

The major challenges in this youthful field of machine learning are building appropriate hardware and making it work. That, so far, has kept it the domain of the software engineer. As the situation continues to stabilise this is going to become the playground of statisticians and analysts.

Or to put it another way - if you compare any field to software engineering, the problem is that other disciplines have a much harder time debugging things. Software is almost unique in that debugging is cheaper and quicker than building things right the first time.

Re: Why is machine learning 'hard'? (2016)

#49

Post by jvans »

Machine learning systems are hard because your system can be badly under performing but still doing objectively great. I've seen systems that produce amazing improvements in core business metrics and then years later some subtle bug in the labeling process is accidentally uncovered and fixing it boosts performance by an additional 20%.

A ton of time in ML is spent on minimizing the surface area for bugs because it can be difficult to even know they exist.

Re: Why is machine learning 'hard'? (2016)

#50

Post by nerdponx »

jvans wrote: post #49

Machine learning systems are hard because your system can be badly under performing but still doing objectively great. I've seen systems that produce amazing improvements in core business metrics and then years later some subtle bug in the labeling process is accidentally uncovered and fixing it boosts performance by an additional 20%. A ton of time in ML is spent on minimizing the surface area for bugs because it ca…

It's hard because there's rarely one single correct solution, design, answer, etc. It's hard in the same way as any other open-ended research work is hard. It will never not be hard in that sense.
Post reply on HN