Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine learning is still too hard for software engineers

nyckel.com

Post reply on HN
Print view View original

71–80 of 154 posts

Re: Machine learning is still too hard for software engineers

#71

Post by hooande »

habitue wrote: post #56
fxtentacle wrote: post #29

"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure th…

This goes a little too far. For traditional ML, sure, you need lots of deep statistical knowledge. But the fact is that deep learning is different: it's mostly a black box. no one understands exactly what they're doing, how they're biased, and how exactly these models understand things differently than humans, whether you have a PhD in statistics or not. Because of that, doing deep learning consists of a bunch of cob…

Not sure why you're making a distinction between ML and deep learning here. both of them can be black boxes. Calculating the area of a square can be a black box if all you know is how to plug numbers into the formula.

A big part of machine learning is looking at weights and outputs to make sure the results are sane and that you have an understanding of what's going on. This is true no matter what algorithm you use to make predictions.

Re: Machine learning is still too hard for software engineers

#72

Post by binarymax »

Thaxll wrote: post #46

I view ML the same as crypto, a fad that will go away in the future and remains in certain nich realms.

ML has already added actual value to so many industries. While the AGI fantasy may never happen and we may even see another AI winter, the existing products and solutions that you use every day are powered by ML.

Re: Machine learning is still too hard for software engineers

#73

Post by habitue »

etangent wrote: post #63
habitue wrote: post #56

Earlier quoted context omitted.

This goes a little too far. For traditional ML, sure, you need lots of deep statistical knowledge. But the fact is that deep learning is different: it's mostly a black box. no one understands exactly what they're doing, how they're biased, and how exactly these models understand things differently than humans, whether you have a PhD in statistics or not. Because of that, doing deep learning consists of a bunch of cob…

Even so, there are procedures, protocols, and best practices for working with (and validating) black boxes, acquiring which may require time, skill, and patience.

Sure, but it's finite, reasonably circumscribed, and honestly not that mathy.

I mean, even the example given by the OP about the tanks is super well known (apocryphal[0]) and doesn't require math knowledge to avoid. You just have to have heard of this kind of failure mode

[0] https://www.gwern.net/Tanks

Re: Machine learning is still too hard for software engineers

#74

Post by hinkley »

fxtentacle wrote: post #29

"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure th…

"For a software engineer, the hardest thing about developing Machine Learning functionality should be finding clean and representative ground-truth data, but it often isn’t."

Which is so much bullshit. The hardest thing is validating your hypotheses, which machine learning turns into a black box. When we have coworkers who insist on operating on wishful thinking we try to maneuver them out of a job. Except every 10-15 years when the built up pressure of fads overwhelms reason and we all get stupid for a generation (which in software is about five years).

The things that started as AI that we don’t call AI anymore, and don’t lump in with AI when discussing successes or failures? It’s because they can be explained in plain English and implemented without much or even any special jargon that marks it as anything more than exceptionally clever Logic.

Re: Machine learning is still too hard for software engineers

#75

Post by disgruntledphd2 »

fxtentacle wrote: post #40
glouwbug wrote: post #32

Earlier quoted context omitted.

Resume driven development is real. Who wants a crud app on their resume when they can have a crud + ML app on their resume? I remember back in like 2016 recruiters devoured anyone with the slightest bit of ML experience on their resumes: it fed back into the ego of developers, and suddenly everyone was an ML expert who could do no better than load a JSON of data and import keras. What a strange trip that time was

I feel like we're seeing the next round of that now with all those "build your own advanced Deep Learning AI in 3 simple steps" websites. Clarifai, Amazon Rekognition, Google AutoML Vision, Nykel (the article here), Amazon Comprehend, Google AutoML Natural Language, MonkeyLearn, Lateral, BigML, Azure ML, Lobe, DataRobot, Rapidminer, Dataiku ... Did I forget anyone? EDIT: H2O’s Driverless AI, Floyd, AWS SageMaker, Dat…

Snowflake is not like the others in that list, as they just provide a pretty good SQL dialect over cloud storage, and a tolerable UI and API access, for a pretty indeterminate price (who knows how much a credit is worth this quarter?).

Personally, if there's ever a downturn, I plan to play Snowflake sales people off each other and get enough credits to last me a lifetime ;)

Re: Machine learning is still too hard for software engineers

#76

Post by jldugger »

xavxav wrote: post #59

Sort of hijacking, but I've always wondered: Where are our 'neural binutils'? I want to be able to compose these tools like I would random unix ones: Something like 'Identify album covers in this image | extract the text in said covers | spotify api'. It seems like there are so many breakthrough models but both due to technical (size/compute) and industrial ($$$) concerns they remain out of reach for random devs, let…

> Where are our 'neural binutils'?

Assuming you mean coreutils. binutils is for managing/inspecting binary executables.

But to your point: there were two key innovations and criteria of UNIX pipelines: a common and understandable data format, and writing programs to send and receive anonymous data. Crucially, the input and output formats were the same: plain text, separated by newlines.

In contrast neural networks are applied to a variety of data formats. Images, video, audio, text, social networks etc. each with their own encoding into something an NN can work with, with varying dimensions, features, metadata etc. So it doesn't make sense to bundle them as 'neural utils' but rather utils along whatever pipeline already exists, like GraphicsMagick. Which does leave a huge blind spot for the domain transforms like text recognition.

If you stay within the AI ecosystem, you _can_ set up reusable layers for tensorFlow, but typically you cant swap out something in the middle without retraining all layers below it. Which you might treat as a violation of the anonymity criterion, since the behavior / performance of a layer relies on the specific behavior of those above it.

Re: Machine learning is still too hard for software engineers

#78

Post by vladoh »

fxtentacle wrote: post #29

"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure th…

While driving, do you understand how the engine ECU of your car computes how much fuel to inject into the cylinder, how the ECUs distribute the power, and the braking force on individual wheels, or how your rear wheel steering calculates the turning angle of the rear wheels based on your seed and steering input?

You don't need to know most of the details how your car works in order to drive. You need much more knowledge to build one, yes, but not to drive.

There are different levels of abstractions and depending on your problem you need to understand them only up to a certain level. And different people have different problems to solve.

In most real-world problems today, the difficult part is indeed the data, not the underlying math of the activation function, loss function, or optimizer. Just Google "data-centric AI Andrew Ng" to read more on the topic from one of the most well-known people in ML.

Re: Machine learning is still too hard for software engineers

#79

Post by disgruntledphd2 »

uoaei wrote: post #48
jstx1 wrote: post #39

Earlier quoted context omitted.

It's not an assumption, that's my main point - you need both skills in the same person. And it's true everywhere including in the biggest companies.

Any good tech lead will remove that need, so no, still not good general advice.

Regardless of how low you get your communication overhead, it still exists. It's rare to find people who can both run and test all the infrastructure and model code, and notice that the transformation you apply on line 34876 of file foobar_now_with_added_ml.py is statistically inappropriate for your problem.

That's not even to mention the really hard part, selecting a good outcome variable and appropriate ways to measure the performance of your system once it hits prod.

You can definitely split this stuff between people, but it gets super-linearly harder as you add more people, so it's really incredible to find people who can do both (and honestly, there aren't that many of them (I'd like to say us, but I'm probably not there yet)).

Re: Machine learning is still too hard for software engineers

#80

Post by habitue »

igorkraw wrote: post #68
habitue wrote: post #56

Earlier quoted context omitted.

This goes a little too far. For traditional ML, sure, you need lots of deep statistical knowledge. But the fact is that deep learning is different: it's mostly a black box. no one understands exactly what they're doing, how they're biased, and how exactly these models understand things differently than humans, whether you have a PhD in statistics or not. Because of that, doing deep learning consists of a bunch of cob…

The "deep learning is a black box" meme is about 5 years past it's due date. It's not as tight as for convex models but we do understand what's going on inside, just not perfectly yet.

I think we're talking about different levels of understanding. For things like convex optimization, we have optimality results. For deep learning we have "try to stop training early because it tends to get overfit if you run it too long", "increase the number of parameters in the transformer to magically get uncanny impressions of written text out". These are not the same kind of understanding.

If I hand you a 175B parameter language model, are you really contending we know what's going on in there? At a mechanical level, sure, it's tensor products and activation functions, but that's like saying we know how human brains work because the standard model is very predictive.

Post reply on HN