No shit. The majority of students I met in the Computer Science weren't skilled in math but took it as a requirement to code. Why would you expect them to just latch onto Machine Learning?
Machine Learning Is Still Too Hard for Software Engineers
51–60 of 112 posts
Re: Machine Learning Is Still Too Hard for Software Engineers
#52ML is one of the easiest fields out there. When I learned it I was actually turned off by how simplistic the concept was. Of course let me preface to say that it's hard to develop the intuition and skill in the same way learning to skateboard is hard. But conceptually it's easy and very possible for almost anyone. The whole thing is just curve fitting. Literally finding some best fit curve across a series of points.…
> Douglas Hofstadter had this whole theory of consciousness and when he found out that an LLM was a simple feed forward network with no feedback loops he went into a crisis. Basically his whole theory in GEB was wrong, according to him.
LLMs are not conscious. The training process for an LLM is not a feed forward network. If we were going to try to fit the idea of consciousness a la humanity (which is really the only fully 'conscious' creature we know of) into LLMs, then 'running' an LLM is identical to cloning a frozen human, thawing it, firing some neurons, reading the result and then destroying the clone.
A better argument for actual consciousness would come from the training process, but that itself is also dubious. It's unlikely consciousness is an emergent phenomenon. Or rather, such a claim is extraordinary and would require an extraordinary amount of proof, which GEB does not provide, sorry.
Re: Machine Learning Is Still Too Hard for Software Engineers
#53Earlier quoted context omitted.
> There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. This is my realization too. I think ML for SWE courses should focus on "translation" first. Like by "kernel" they mean this specific thing, not the normal meaning of kernel. This is similar to other fields like finance (which I'm working on). After you learn the language, it's actually no…
What's the normal meaning of 'kernel'? EDIT: Hacker news won't let me respond, but the answers below all seem to be because the original meaning has been lost on everyone. In English, the word 'kernel' means 'core'. An OS kernel is the core of an operating system. In linear algebra, the kernel of a matrix (or a linear transformation, same thing) is the set of vectors it maps to zero, which is also in a sense the 'cor…
Well, I'm not actually sure which one they're referring to but there is two meanings that I know of. There's the Software Developer Kernel which is the core of an operating system, and there's the Linear Algebra Kernel which is the null space of a map. Most of us here are familiar with the operating system type but the linear algebra kernel describes the set of vectors which are sent to the null vector (0) by your transformation/map/operation. It's very useful in understanding what exactly your operator does and how it changes a space.
Re: Machine Learning Is Still Too Hard for Software Engineers
#54There's a big gap between training an algorithm on a toy problem, vs building a useful product.
Software engineers often are missing key skills. They can learn them, but won't automatically get them in their traditional training.
First, measuring success. Actually telling how well a production system is doing is tricky. There's an art to developing metrics that tell you if an ML system is delivering value, and a lot for engineers don't have the metric design skills. Often to productionize an ML system, you need a bunch of proxy metrics and a pretty good backtesting setup. This will often depend on the specific problem, and the skill of it is something you won't get in a standard software setting.
Engineers - and especially designers - also struggle with edge cases when things go off the happy path. It's often easy to make an ML prototype that works in 90% of cases, and get a project started - but a nightmare to solve enough the edge cases for a production grade system. Finding and papering over and designing around all those edge cases effectively can require a deep bag of tricks a pure software engineer won't have.
Finally there's a struggle with tactics and culture. A lot of the bread and butter tactics of high performing software delivery are the opposite of what you need for ML projects. E.g. In high velocity frontend work you want to lock a design early, and your designer can probably do a lot of iteration before engineering starts. In ML projects you want to keep the design floating and low fidelity, as you prototype, and lock it late in the project.
So many development tactics, and cultural patterns, that lead to high performing software teams, in a SaaS setting, say, are anathema to ML projects.
Re: Machine Learning Is Still Too Hard for Software Engineers
#55Earlier quoted context omitted.
What's the normal meaning of 'kernel'? EDIT: Hacker news won't let me respond, but the answers below all seem to be because the original meaning has been lost on everyone. In English, the word 'kernel' means 'core'. An OS kernel is the core of an operating system. In linear algebra, the kernel of a matrix (or a linear transformation, same thing) is the set of vectors it maps to zero, which is also in a sense the 'cor…
The bit of code at the center of an operating system that mediates access between userspace code and the hardware. Whereas in CUDA programming, a kernel is just the code running on the device. When I first heard kernel in relation to CUDA programming, I expected it to be a) the actual OS kernel, and then b) something akin to a GPU driver. I can see how NVIDIA got there, but it's not immediately obvious if you're comi…
Kernel is also used in convolutions... Ugh
Re: Machine Learning Is Still Too Hard for Software Engineers
#56did you mean deep learning?
Re: Machine Learning Is Still Too Hard for Software Engineers
#57Earlier quoted context omitted.
> It felt like I needed a PhD and hardcore math skills I don't know if this helps more or make it worse. But I have both and getting the same feelings all the time. But basically you just need good statistics and linear algebra knowledge, and you will be fine (on the math side). > there’s already so many people much more smarter and advanced for me. Why even bother? That's the very definition of imposter syndrome put…
99% of the time people say linear algebra is required for something, they mean knowledge basic operations and properties of tensors more than actual "algebra". I found this when doing computer graphics. Is that true as well here?
Basic operations and properties of tensors are exactly what is taught in linear algebra, in addition to (in my experience) more accessible proofs. More or less through singular value decomposition and/or least squares.
The core algorithms all build on top of each other. The `algebra` part of linear algebra refers to a `field`, but it might as well also be called arithmetic of tensors.
Re: Machine Learning Is Still Too Hard for Software Engineers
#58As a software engineer, I disagree. Caveat, I haven't studied traditional ML and just went straight to DL. There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. One thing that actually really bothers me is how much libraries (e.g. huggingface) are just config files masquerading as programming. It is just a class with 50 parameters and takes ab…
> we just need some core work horse libraries
But these do exist: plain Jax or Pytorch only give you basic linear algebra, differentiation and some basic layers. And there's a plethora of more or less advanced libraries that add specific functionality, for example torch geometric for graph data and lightning to reduce boilerplate.
Re: Machine Learning Is Still Too Hard for Software Engineers
#59As a software engineer, I disagree. Caveat, I haven't studied traditional ML and just went straight to DL. There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. One thing that actually really bothers me is how much libraries (e.g. huggingface) are just config files masquerading as programming. It is just a class with 50 parameters and takes ab…
> There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. This is my realization too. I think ML for SWE courses should focus on "translation" first. Like by "kernel" they mean this specific thing, not the normal meaning of kernel. This is similar to other fields like finance (which I'm working on). After you learn the language, it's actually no…
Of course you mean the "normal" meaning of a kernel pertaining to Linear Algebra, right?
https://en.wikipedia.org/wiki/Kernel_(linear_algebra)
Of course I'm kidding. It's one of those terms that many fields adopt and give it completely different meaning. There's no "normal" meaning of `kernel`.
Re: Machine Learning Is Still Too Hard for Software Engineers
#60As a software engineer, I disagree. Caveat, I haven't studied traditional ML and just went straight to DL. There is a lot of jargon and you do have to sit down and learn how things work, but once you do, deep learning is fairly simple. One thing that actually really bothers me is how much libraries (e.g. huggingface) are just config files masquerading as programming. It is just a class with 50 parameters and takes ab…
[dead]