Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning for Developers

xyclade.github.io

Post reply on HN
Print view View original

51–60 of 99 posts

Re: Machine Learning for Developers

#51

Post by mistermann »

zxcvvcxz wrote: post #25

Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.

9 times out of 10 it will be a clueless manager that read in Gartner that ML is the next big thing so they'll put some programmers on it, they'll click their heels together at learning something new, nothing will come out of it, except they funded the platform for the much smaller group of people who'll actually use it for something useful. Win win.

Re: Machine Learning for Developers

#52

Post by sotojuan »

mystique wrote: post #44
zxcvvcxz wrote: post #25

Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.

A decent CS undergrad degree decade ago included abstract math concepts. I took Engineering math, Information Theory, Numerical analysis, Probability, Simulation in my sophomore and Junior years. NLP and AI were electives in Senior year. As a Junior, we were building toy programs that do Operations research type of work - solving linear equations via various matrix operations, design optimal queue processes based on…

Bad CS school student here, we don't take any Math aside from a very basic "discrete structures" class, which is simplified discrete structures :/ Wish we did more math...

Re: Machine Learning for Developers

#53

Post by aficionado »

jorgemf wrote: post #46
aficionado wrote: post #38

Earlier quoted context omitted.

I'm a little bit worried. At least at the same level as when I see a bunch of developers compiling programs without much understanding of what an LL(k) parser does, or how a pushdown automaton works, or what a Turing machine is. I usually feel the same every time I see an elevator without a liftman, don't compute a square root by hand, or hear about Google self-driving cars.

The difference is that the software or the elevator will work but the statistical model is wrong and doesn't work. It is like the elevator only lift people above 120 and below 90 and for the others it just don't work or take you to the wrong floor.

How many elevators were failing today? how many were fixed today? and how many have been monitored in real-time to get fixed as they fail? Likewise, statistical models can be monitored and fixed automatically, and, of course, even so they will fail from time to time like elevators do.

Re: Machine Learning for Developers

#54

Post by aficionado »

argonaut wrote: post #49
aficionado wrote: post #38

Earlier quoted context omitted.

I'm a little bit worried. At least at the same level as when I see a bunch of developers compiling programs without much understanding of what an LL(k) parser does, or how a pushdown automaton works, or what a Turing machine is. I usually feel the same every time I see an elevator without a liftman, don't compute a square root by hand, or hear about Google self-driving cars.

It's different. In ML the model, the analysis, and the insights, are the product. In general software engineering, your compiler is not your product.

Don't think so. The data and how it's represented is the code. ML is the compiler.

Re: Machine Learning for Developers

#55

Post by tel »

zxcvvcxz wrote: post #25

Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.

Not really. What will have to happen is a readjustment for recognizing that poorly designed models just don't work very well.

Re: Machine Learning for Developers

#56

Post by JohnnyWang »

zxcvvcxz wrote: post #25

Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.

I always assume that if a person want to do much better in ML, he will try to learn stuffs he didn't know before, like statistics, math, etc. Everyone's knowledge is limited, but that doesn't limit what people can do, just need to learn more, I guess.

Re: Machine Learning for Developers

#57

Post by tel »

zxcvvcxz wrote: post #25

Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.

Not really. What will have to happen is a readjustment for realizing that many models, especially made by people with little experience and training, will be wrong.

Right now it's the glory days of ML when nobody much has the ability to judge success. Unlike software engineering broadly, where these glory days just keep going, ML is all about measuring success. People will detect failures.

The real risk is when people systematically underestimate the risk like the copula thing occurring with the subprime market. That was anything bug untrained people using models—they would not have been as dangerous as they were if they weren't so damn good to begin with. This is a robustness failure, not a poorly trained workforce failure.

Re: Machine Learning for Developers

#58

Post by argonaut »

aficionado wrote: post #54
argonaut wrote: post #49

Earlier quoted context omitted.

It's different. In ML the model, the analysis, and the insights, are the product. In general software engineering, your compiler is not your product.

Don't think so. The data and how it's represented is the code. ML is the compiler.

This is overly pedantic. The code is doing machine learning. In order to write and understand the code you have to understand the machine learning algorithms. Before you even choose a model and tune the parameters you have to know how the parameters interact and how different models work.

Re: Machine Learning for Developers

#59

Post by geomark »

hugh4 wrote: post #42
wyldfire wrote: post #34

Earlier quoted context omitted.

> those firms whose models don't reflect reality will blow up, I'm picturing one of those dystopic films/novels where the main character is deleted/fired/jailed as a result of an algorithm error. Yes, in real life the trends will overcome the bad models. But just think of the potential consequences for harm on an individual basis!

Isn't that just the plot of Minority Report? Think of the possibilities of machine learning for detecting precrime!

It's already here. See http://www.predpol.com/

Re: Machine Learning for Developers

#60

Post by rdtsc »

zxcvvcxz wrote: post #25

Is anyone else at least a bit worried about a bunch of developers running around doing "machine learning" without much understanding of mathematics and probability? E.g. consider the creation of fragile models that overfit data being used in finance, infrastructure, medicine, etc.

Nah. The only topic I would be worried about is cryptography, when used in a non-learning context. That has a high potential to cause harm. Otherwise with machine learning, I don't see how it is necessarily more dangerous than any other software -- databases, network protocols and so on...
Post reply on HN