Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning Is Still Too Hard for Software Engineers

nyckel.com

Post reply on HN
Print view View original

71–80 of 112 posts

Re: Machine Learning Is Still Too Hard for Software Engineers

#71

Post by ronald_raygun »

testless wrote: post #66

I am mathematician by training. The algorithms relied mostly on undergrad level mathematics when I took some courses six years ago, not easy, but I think there are harder algorithms. I think it is qualitatively different from programming, because you try to find a reasonable good fit for data you know to guess new future data. Classical programming relies on rules and decisions. ML is closer to numerics, statistics,…

You can easily get into territory that is harder than undergrad. Like it’s completely possible you’d need something like rejection sampling, functional time series, Jeffrey’s priors, etc

Re: Machine Learning Is Still Too Hard for Software Engineers

#72

Post by mrits »

thealienthing wrote: post #14

I feel so validated by this article. I took two semesters of machine learning electives for my CS masters and feel nearly as ignorant and mystified as when I started. I worked so hard to create something useful and at the end of the day, my work felt like it was 96% example code with modifications hacked in to make it work. And in the end it was still terrible! At least now I know what people are talking about when d…

I'd be surprised if it doesn't suffer the same fate as graphics programming. The lower level stuff is what brings in a lot of talent but the producers often have little knowledge of how things work but just wire together some libraries in a GUI.

Re: Machine Learning Is Still Too Hard for Software Engineers

#73

Post by lordgroff »

thealienthing wrote: post #14

I feel so validated by this article. I took two semesters of machine learning electives for my CS masters and feel nearly as ignorant and mystified as when I started. I worked so hard to create something useful and at the end of the day, my work felt like it was 96% example code with modifications hacked in to make it work. And in the end it was still terrible! At least now I know what people are talking about when d…

Well that's because ML isn't really software engineering. Unfortunately, it is also software engineering as I often have to remind my colleagues coming from algebra / econometrics / statistics sides who are happy to shove all kinds of horrible code in.

Re: Machine Learning Is Still Too Hard for Software Engineers

#74

Post by tdiff »

godelski wrote: post #48
AznHisoka wrote: post #2

For me the hardest part of learning ML was getting over imposter syndrome. It felt like I needed a PhD and hardcore math skills. That’s what made me so hesitant in learning it. I thought: there’s already so many people much more smarter and advanced for me. Why even bother? It wasn’t until I was “forced” to learn it to solve a problem I was facing, that I realized ML is just like any other engineering topic - whether…

> For me the hardest part of learning ML was getting over imposter syndrome. It felt like I needed a PhD and hardcore math skills ABD (all but PhD dissertation) here with strong math skills. I get the imposter syndrome, but let me absolutely assure you that the community at large does not have strong math skills. I routinely talk to people doing diffusion research that don't know what covariance is or pdf. People fro…

Pdf is a file format

Re: Machine Learning Is Still Too Hard for Software Engineers

#75

Post by godelski »

anon291 wrote: post #29
carom wrote: post #16

As a software engineer, I disagree. Caveat, I haven't studied traditional ML and just went straight to DL. There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. One thing that actually really bothers me is how much libraries (e.g. huggingface) are just config files masquerading as programming. It is just a class with 50 parameters and takes ab…

The post is not talking about 'traditional ML' but rather only 'DL' (i.e. neural networks). One of the problems I see here is that it seems to me that the math education of many CS grads is woefully lacking. Indeed, deep learning math is basically senior-high-school level calculus. Backpropagation is a straightforwards application of the chain rule. There is really no surprising thing or deep insight. The math for de…

> Indeed, deep learning math is basically senior-high-school level calculus.

Actually I find this claim problematic and incorrect. Yes, many parts of ML only require multivariate calculus[0] and those are the parts that most people are exposed to. BUT that doesn't mean there isn't a lot of math hiding around in the background. Even understanding something like activation functions take much higher level math as we got to get into topics such as topology, metric theory, and high dimensional statistics. You don't need this understanding to build models (as you also conclude) and most researchers don't understand much of this either tbh. But I think we need to be clear about these distinctions as model evaluation gets insanely complex. Complexity in evaluation only increases as our performance increases and a major problem we face today is we are marginalizing out important information. I'm sure that anyone that hacks around with LLMs or diffusion will understand how leaderboards are often noisy and how picking top models does not guarantee top performance (and why datasets get continually added). It's because evaluation requires more than boiling down performance to a single number. Here, math skills become extremely important.

[0] btw, many will not find this even offered at their high school. In fact, this is often upper level math for many STEM undergraduates and even an elective in my Uni's undergrad CS program. So let's not be so belittling. Shame the education systems instead of people lacking opportunities.

Re: Machine Learning Is Still Too Hard for Software Engineers

#77

Post by bugglebeetle »

thealienthing wrote: post #14

I feel so validated by this article. I took two semesters of machine learning electives for my CS masters and feel nearly as ignorant and mystified as when I started. I worked so hard to create something useful and at the end of the day, my work felt like it was 96% example code with modifications hacked in to make it work. And in the end it was still terrible! At least now I know what people are talking about when d…

> 96% example code with modifications hacked in to make it work.

This is 96% of how ML is used in practice by companies. Most parameter optimization should be automated by whatever library you’re using, beyond basic sanity checking. The challenging parts are creating high-quality training data and deploying the models efficiently at scale.

Re: Machine Learning Is Still Too Hard for Software Engineers

#78

Post by metanonsense »

In my opinion, current SOTA machine learning / deep learning is actually quite simple on a conceptual level. January last year, I decided to give it one year to understand transformers on a "I could totally code that" level. There were so many good video tutorials and texts that after 2 months there wasn't so much left to learn. In particular, the math did not feel much more difficult than what I learned in high school (and much easier than what we did in university). I think this is a particularity of those DL approaches that don't need much more than some (higher dimensional) linear algebra and calculus. There are other AI or ML approaches that I cannot even grasp on a superficial level, but fortunately (from a learner's perspective) DL has completely eaten machine learning, for the time being.

Nevertheless, that entire field depends so much on complex heuristics and subtle optimizations that I completely understand that learning and getting an intuition for those details takes a long time (and is more akin to black magic). The development experience is absolutely horrible. Debugging a model takes such a long time, is rather expensive, and observability is absolutely dismal (at least it was a year ago). It really felt like debugging a program into existence by staring at graphs and retrying infinitely many times.

Re: Machine Learning Is Still Too Hard for Software Engineers

#79

Post by beiller »

lordgroff wrote: post #73
thealienthing wrote: post #14

I feel so validated by this article. I took two semesters of machine learning electives for my CS masters and feel nearly as ignorant and mystified as when I started. I worked so hard to create something useful and at the end of the day, my work felt like it was 96% example code with modifications hacked in to make it work. And in the end it was still terrible! At least now I know what people are talking about when d…

Well that's because ML isn't really software engineering. Unfortunately, it is also software engineering as I often have to remind my colleagues coming from algebra / econometrics / statistics sides who are happy to shove all kinds of horrible code in.

What I've found in reality is that machine learning is 99% data cleaning scripts and 1% the part you're talking about. I've also seen the heavy duty statistics people writing data cleaning python scripts which probably leads to a lot of frustrations :)

Re: Machine Learning Is Still Too Hard for Software Engineers

#80

Post by beijbom »

blackbear_ wrote: post #58
carom wrote: post #16

As a software engineer, I disagree. Caveat, I haven't studied traditional ML and just went straight to DL. There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. One thing that actually really bothers me is how much libraries (e.g. huggingface) are just config files masquerading as programming. It is just a class with 50 parameters and takes ab…

> we just need some core work horse libraries But these do exist: plain Jax or Pytorch only give you basic linear algebra, differentiation and some basic layers. And there's a plethora of more or less advanced libraries that add specific functionality, for example torch geometric for graph data and lightning to reduce boilerplate.

(I'm one of the Nyckel co-founders)

Chiming in to say that this precisely our observation. The existing ML/DL libraries are not bad as far as those types of things go. In fact, Pytorch is an amazing library IMO. Especially compared to TensorFlow, Caffe and the stuff that came before that.

But like George points out in the article, unlike "traditional" software, ML requires iteration, data management, monitoring, specific infra reqs, and so on. So our take was that libraries would never be enough, hence the SaaS offering.

Post reply on HN