Most CS grads coming onto the engineer market now will have ML exposure through their chosen college courses. As this wave of knowledge makes its way through the industry, the value of ML specialist knowledge will decline, especially as off-the-shelf pre-trained models improve. Very few companies then will be able to justify the luxury of a dedicated in-house data science/ML engineering team. In other words, most software engineers will be ML engineers, in the same way most software engineers today are Docker/cloud proficient engineers.
Machine learning is still too hard for software engineers
81–90 of 154 posts
Re: Machine learning is still too hard for software engineers
#82"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure th…
Who cares if you overfitted? See, the model has 100% success rate vs the training set! Who cares if it denies bail to minorities or hits a few pedestrians from time to time? The problem isn’t that ML is too hard, it’s that it’s too easy. Crazy people keep connecting ML to systems that matter- that have real, irreversible impact to humans- and they don’t understand it. I wish ML were 1000x harder/more expensive to int…
This is exactly right.
I've seen it time and time again: Team has a black box ML/AI solution to a "problem." Team wants to eke out better P/R or deal with some complex edge cases. But team's problem is fundamentally ill-posed and no amount of hacking or kludges will actually produce the success criterion that they need.
The problem is the accessibility to these tools, which in many times has led folks to neglect the subject matter expertise required to effectively apply them in the first place. At least as these tools catch on in popularity in myriad problem domains, there will be a new generation of subject matter / domain experts who are familiar with them, and we'll probably jump over this hurdle.
Re: Machine learning is still too hard for software engineers
#83We made an experimental foray into the ML space in attempts to accelerate authoring of SQL queries.
After about a week of reading through literature and playing with open ai, it became pretty obvious we were still super far away from being able to build something the business could actually find value in.
My problem scales horribly with the ML training angle we have today, because it's the super complex one-off queries we need the most help with, not the simple ones we can anticipate and train against.
What we need is actual intelligence for many of our problems. Things like subjective criteria are important to us. Realizing maybe a recursive query is a fair compromise to reduce a 400 line monster to 30 lines. Assuming the 400 "looks nasty", that is. I guess you could train that bit too, but then your solution space gets even more impossible to target.
Re: Machine learning is still too hard for software engineers
#84No, this article is wrong on the first sentence. The hardest part of machine learning _is_ in fact curating quality data.
That's the essential complexity. The OP is correct that there's far more accidental complexity involved than seems strictly necessary. In an ideal world, I wouldn't have had to spend the last few years learning software engineering in order to be a better data scientist.
Re: Machine learning is still too hard for software engineers
#85So, I went directly into data science after an econ degree, worked there for 2 years and then transitioned to SWE (at startups). First, I am 100% certain ML will become a part of the standard SWE toolkit (just like apis, docker, sql, etc..).
However, to the relative "hardness" I would say ML currently is much less things but they can be really hard to get your head around (like starting to think in embeddings and vectors), and many APIs are wonky, because they are new. And the whole data science workflow space it in an early spot (we are still missing a "create-data-app", simmilar to the create-react-app.
Re: Machine learning is still too hard for software engineers
#86Earlier quoted context omitted.
Even so, there are procedures, protocols, and best practices for working with (and validating) black boxes, acquiring which may require time, skill, and patience.
Sure, but it's finite, reasonably circumscribed, and honestly not that mathy. I mean, even the example given by the OP about the tanks is super well known (apocryphal[0]) and doesn't require math knowledge to avoid. You just have to have heard of this kind of failure mode [0] https://www.gwern.net/Tanks
> You just have to have heard of this kind of failure mode
Yes exactly. You have to be aware of it, you have to know what it entails and what can cause it and how to diagnose and fix it.
That’s the other half of the domain knowledge, and just “autoML-ing it” or following some set of prescribed steps won’t necessarily get you that solution.
Re: Machine learning is still too hard for software engineers
#87The challenge for the engineers at our AI startup is that deterministic testing paradigms don't adapt well to probabilistic models that are continually being retrained. As a scientist is hard to convey the acceptable range of variance and often the random change of individual predictions at the decision boundary. It's also hard debug behavioral issues that actually are systematic model failure versus those that are traditional infrastructure bugs. Often times the band aid is that build lookup tables to ensure certain behavior which in turn also underlying issues from being discovered.
Testing paradigms are either too high level or too specific. Recent work on evolving behavioral tests addresses this but it requires more manual effort and interpretation which kinda defeats to point of automated tests.
Re: Machine learning is still too hard for software engineers
#88Coming from a background in computational quantum chemistry, it’s interesting to see all of the people who say ML is “easy” after taking a few online courses and reading some books on data science. If it’s so easy, invent AGI then, since that is the holy grail of machine learning. Most of these people claiming expertise do not have a deep grasp of the mathematical fundamentals required for state of the art research i…
Not to disagree with your point, but your comment really reminded me of the following scene from Breaking Bad: https://www.youtube.com/watch?v=W_dxteeedgs
Re: Machine learning is still too hard for software engineers
#89So, I went directly into data science after an econ degree, worked there for 2 years and then transitioned to SWE (at startups). First, I am 100% certain ML will become a part of the standard SWE toolkit (just like apis, docker, sql, etc..). However, to the relative "hardness" I would say ML currently is much less things but they can be really hard to get your head around (like starting to think in embeddings and vec…
Your career path seems really interesting. Do you have a blog or some kind of work showcase where we can follow your journey and work?
Re: Machine learning is still too hard for software engineers
#90Earlier quoted context omitted.
Aleksa is great. Also check out Jeremy Howard from fast.ai - also no PhD but amazing teacher and contributes actively to research. Chris Olah (Google Brain, OpenAI, etc) go to university at all. PhD definitely not required.
I tried to build on top of fast.ai, and it was very easy to start, but all the hooks and magic in fast.ai 2 just made it extremely hard for me to understand and extend the code. I believe it went in a bad direction.
When tensorflow was the dominate api, fast.ai made a lot of sense. Nowadays pytorch seems to have won and it is easy to use directly and there is PyTorch Lightning if you want things even easier to write.
There are a several videos on youtube where the creator reviews a paper and then implements it from scratch. For example this channel is pretty good: https://www.youtube.com/c/AladdinPersson/playlists