Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning Is Still Too Hard for Software Engineers

nyckel.com

Post reply on HN
Print view View original

101–110 of 112 posts

Re: Machine Learning Is Still Too Hard for Software Engineers

#101

Post by dpig_ »

anon291 wrote: post #31
thanhhaimai wrote: post #30

Earlier quoted context omitted.

> There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. This is my realization too. I think ML for SWE courses should focus on "translation" first. Like by "kernel" they mean this specific thing, not the normal meaning of kernel. This is similar to other fields like finance (which I'm working on). After you learn the language, it's actually no…

What's the normal meaning of 'kernel'? EDIT: Hacker news won't let me respond, but the answers below all seem to be because the original meaning has been lost on everyone. In English, the word 'kernel' means 'core'. An OS kernel is the core of an operating system. In linear algebra, the kernel of a matrix (or a linear transformation, same thing) is the set of vectors it maps to zero, which is also in a sense the 'cor…

Although, funnily enough, we don't use kernel to mean 'core' in the idiom 'kernel of truth.'

Re: Machine Learning Is Still Too Hard for Software Engineers

#102

Post by tnecniv »

thanhhaimai wrote: post #30
carom wrote: post #16

As a software engineer, I disagree. Caveat, I haven't studied traditional ML and just went straight to DL. There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. One thing that actually really bothers me is how much libraries (e.g. huggingface) are just config files masquerading as programming. It is just a class with 50 parameters and takes ab…

> There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. This is my realization too. I think ML for SWE courses should focus on "translation" first. Like by "kernel" they mean this specific thing, not the normal meaning of kernel. This is similar to other fields like finance (which I'm working on). After you learn the language, it's actually no…

Kernel is an overloaded term outside of software engineering. In linear algebra it means the null space, but that has no connection I’ve ever found to kernel methods or kernels more generally in functional analysis.

Re: Machine Learning Is Still Too Hard for Software Engineers

#103

Post by tnecniv »

blackbear_ wrote: post #70
anon291 wrote: post #31

Earlier quoted context omitted.

What's the normal meaning of 'kernel'? EDIT: Hacker news won't let me respond, but the answers below all seem to be because the original meaning has been lost on everyone. In English, the word 'kernel' means 'core'. An OS kernel is the core of an operating system. In linear algebra, the kernel of a matrix (or a linear transformation, same thing) is the set of vectors it maps to zero, which is also in a sense the 'cor…

In a ML context kernel would probably be understood as a way to quantify similarity between different examples: https://en.m.wikipedia.org/wiki/Kernel_method But it was hilarious to see how many other "normal" meanings came up in the comments.

Normal is just the first one you learn.

Hell, in math, normal even has multiple meanings. You have the normal distribution and surface normals for example

Re: Machine Learning Is Still Too Hard for Software Engineers

#104

Post by tomrod »

shortrounddev2 wrote: post #93
tomrod wrote: post #57

Earlier quoted context omitted.

Basic operations and properties of tensors are exactly what is taught in linear algebra, in addition to (in my experience) more accessible proofs. More or less through singular value decomposition and/or least squares. The core algorithms all build on top of each other. The `algebra` part of linear algebra refers to a `field`, but it might as well also be called arithmetic of tensors.

Right but I mean that people say you need linear algebra (e.g: the whole textbook), when in actuality you might need the first 3 chapters

I think I didn't come across clearly. Eigenvalues, SVD, etc are important.

Re: Machine Learning Is Still Too Hard for Software Engineers

#105

Post by mov »

beijbom wrote: post #80
blackbear_ wrote: post #58

Earlier quoted context omitted.

> we just need some core work horse libraries But these do exist: plain Jax or Pytorch only give you basic linear algebra, differentiation and some basic layers. And there's a plethora of more or less advanced libraries that add specific functionality, for example torch geometric for graph data and lightning to reduce boilerplate.

(I'm one of the Nyckel co-founders) Chiming in to say that this precisely our observation. The existing ML/DL libraries are not bad as far as those types of things go. In fact, Pytorch is an amazing library IMO. Especially compared to TensorFlow, Caffe and the stuff that came before that. But like George points out in the article, unlike "traditional" software, ML requires iteration, data management, monitoring, spec…

That's a super interesting article and point of view. So, in your opinion, what's missing?

Re: Machine Learning Is Still Too Hard for Software Engineers

#106

Post by anon291 »

dpig_ wrote: post #101
anon291 wrote: post #31

Earlier quoted context omitted.

What's the normal meaning of 'kernel'? EDIT: Hacker news won't let me respond, but the answers below all seem to be because the original meaning has been lost on everyone. In English, the word 'kernel' means 'core'. An OS kernel is the core of an operating system. In linear algebra, the kernel of a matrix (or a linear transformation, same thing) is the set of vectors it maps to zero, which is also in a sense the 'cor…

Although, funnily enough, we don't use kernel to mean 'core' in the idiom 'kernel of truth.'

How so? Saying something has a 'kernel of truth' means that, while there is some fluff, the core idea holds water.

Re: Machine Learning Is Still Too Hard for Software Engineers

#107

Post by appplication »

beiller wrote: post #79
lordgroff wrote: post #73

Earlier quoted context omitted.

Well that's because ML isn't really software engineering. Unfortunately, it is also software engineering as I often have to remind my colleagues coming from algebra / econometrics / statistics sides who are happy to shove all kinds of horrible code in.

What I've found in reality is that machine learning is 99% data cleaning scripts and 1% the part you're talking about. I've also seen the heavy duty statistics people writing data cleaning python scripts which probably leads to a lot of frustrations :)

I think what may be understated here is that while it’s true that ML is mostly date cleaning, data cleaning is not easy. There are a million little decisions made and it’s rarely clear which ones are most effective. Experimenting with various techniques is great but the iteration times and cost are usually too high to try more than a small handful of approaches.

Re: Machine Learning Is Still Too Hard for Software Engineers

#108

Post by catlover76 »

senseiV wrote: post #37
valine wrote: post #18

Earlier quoted context omitted.

The industry has been consolidating around transformers. I'd say it's getting deeper, not wider.

NLP is not the industry, and a lot of research still goes into other things, like RL I've worked with several transformers competitors, and it def wont stay centralized on them

[dead]

Re: Machine Learning Is Still Too Hard for Software Engineers

#109

Post by kajecounterhack »

fergal_reid wrote: post #54

There's a big gap between training an algorithm on a toy problem, vs building a useful product. Software engineers often are missing key skills. They can learn them, but won't automatically get them in their traditional training. First, measuring success. Actually telling how well a production system is doing is tricky. There's an art to developing metrics that tell you if an ML system is delivering value, and a lot…

> Engineers - and especially designers - also struggle with edge cases when things go off the happy path. It's often easy to make an ML prototype that works in 90% of cases, and get a project started - but a nightmare to solve enough the edge cases for a production grade system. Finding and papering over and designing around all those edge cases effectively can require a deep bag of tricks a pure software engineer won't have.

YMMV. Finding and papering over the things that prevent a model from being deployable can also require a deep bag of engineering tricks that an average ML research scientist does not have. In my personal experience I've seen teams burned by this more often than the other way around.

Re: Machine Learning Is Still Too Hard for Software Engineers

#110

Post by dpig_ »

anon291 wrote: post #106
dpig_ wrote: post #101

Earlier quoted context omitted.

Although, funnily enough, we don't use kernel to mean 'core' in the idiom 'kernel of truth.'

How so? Saying something has a 'kernel of truth' means that, while there is some fluff, the core idea holds water.

I guess I have always interpreted the use of kernel there as meaning something small. But you are probably right.
Post reply on HN