Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine learning is still too hard for software engineers

nyckel.com

Post reply on HN
Print view View original

61–70 of 154 posts

Re: Machine learning is still too hard for software engineers

#61

Post by l33t2328 »

fxtentacle wrote: post #29

"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure th…

> That's how you end up with a "tank detection AI" that in reality just classifies the weather into bright vs. dark.

It seems to me the lack of knowledge was not knowing to use a diverse sample set, not some lack of mathematics knowledge.

Re: Machine learning is still too hard for software engineers

#62

Post by zozbot234 »

habitue wrote: post #56
fxtentacle wrote: post #29

"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure th…

This goes a little too far. For traditional ML, sure, you need lots of deep statistical knowledge. But the fact is that deep learning is different: it's mostly a black box. no one understands exactly what they're doing, how they're biased, and how exactly these models understand things differently than humans, whether you have a PhD in statistics or not. Because of that, doing deep learning consists of a bunch of cob…

It's no different, statistical knowledge is still needed to draw the best possible inferences out of the combination of limited data points and prior general information with varying strengths/confidence levels attached. Not to mention that loosely "black box" methods have a long history of their own in non-parametric and semi-parametric stats, so it's not like deep learning is doing anything radically different.

Re: Machine learning is still too hard for software engineers

#63

Post by etangent »

habitue wrote: post #56
fxtentacle wrote: post #29

"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure th…

This goes a little too far. For traditional ML, sure, you need lots of deep statistical knowledge. But the fact is that deep learning is different: it's mostly a black box. no one understands exactly what they're doing, how they're biased, and how exactly these models understand things differently than humans, whether you have a PhD in statistics or not. Because of that, doing deep learning consists of a bunch of cob…

Even so, there are procedures, protocols, and best practices for working with (and validating) black boxes, acquiring which may require time, skill, and patience.

Re: Machine learning is still too hard for software engineers

#65

Post by spaetzleesser »

I still remember when SQL was a dark science that could only be managed by administrators that were initiated in secret rooms. Nowadays a lot of people use SQL without deep understanding. They do useful stuff but if you are very skilled you can do way more. And SQL experts are often frustrated with all the amateurs that use their database in such a sub-optimal way.

I expect the same for ML. The tooling will improve until regular developers can use it without understanding the fundamentals. They will do useful things but some skilled people will be able to do way more advanced stuff.

That's pretty much the path of all technologies. They get simplified to a level where they useful for a lot of people but some experts will be able to get way more out it. And usually the experts look down on the amateurs.

Re: Machine learning is still too hard for software engineers

#66

Post by tluyben2 »

fxtentacle wrote: post #29

"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure th…

Unfortunately there are a shitload of ML ‘experts’ out there who do not know what they are doing but still get results that are good enough to not get fired and receive copious amounts of money every year. These tools help doing that; companies generally don’t see the difference anyway and they don’t know how to set or evaluate KPIs on these ventures; they don’t even know what or why they are asking for; they just know they need to show progress with AI to not become obsolete.

Re: Machine learning is still too hard for software engineers

#68

Post by igorkraw »

habitue wrote: post #56
fxtentacle wrote: post #29

"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure th…

This goes a little too far. For traditional ML, sure, you need lots of deep statistical knowledge. But the fact is that deep learning is different: it's mostly a black box. no one understands exactly what they're doing, how they're biased, and how exactly these models understand things differently than humans, whether you have a PhD in statistics or not. Because of that, doing deep learning consists of a bunch of cob…

The "deep learning is a black box" meme is about 5 years past it's due date. It's not as tight as for convex models but we do understand what's going on inside, just not perfectly yet.

Re: Machine learning is still too hard for software engineers

#69

Post by binarymax »

This line "Set up low-latency, elastic, highly-available, and cost-effective inference close to your data." is the problem I've found that annoys everyone - Software Eng don't understand MLOps, Data Scientists don't understand systems programming, and everything ends up costing way too much money and taking way too much time.

Even with a magic API the latency still isnt good enough, so the choice is often entire ML solutions like the OPs product, months of development time, or a really expensive container.

Shameless plug since my Show HN went completely ignored earlier today ;) ...but that's why i built this: https://news.ycombinator.com/item?id=30428664

Re: Machine learning is still too hard for software engineers

#70

Post by abc_lisper »

mrfusion wrote: post #30
deepsun wrote: post #19

I've dived into ML (and DL) with 17 years of software development experience. I'd say it's much easier than software. Yes, there's A TON to learn and experiment with, but still much less than with software. I was able to feel confident enough after just 1.5 years learning and kaggling, and passed easily ML interviews to SF Bay Area companies (hint -- all data science people are extremely glad to see software experien…

Can you share how you switched into ML? How and what did you study? Any tips?

Yeah, would appreciate if parent answers this.
Post reply on HN