Machine Learning Is Still Too Hard for Software Engineers

For me the hardest part of learning ML was getting over imposter syndrome. It felt like I needed a PhD and hardcore math skills. That’s what made me so hesitant in learning it. I thought: there’s already so many people much more smarter and advanced for me. Why even bother?

It wasn’t until I was “forced” to learn it to solve a problem I was facing, that I realized ML is just like any other engineering topic - whether it’s devops or data engineering. You just need motivation, some patience and ideally a project/problem that you can solve while learning all this stuff.

it's not anymore surprising than realizing graphics programming requires you to have a good grasp on linear algebra.

the math is how it's done, it's statistics all the way down. If you don't have a good grasp on statistics you're not going to fully grok ML.

Do you have recommendations for good books that are not too mathematical, written for the average developer?

Andrew Ng's course for theoretical underpinnings, and Jeremy Howard's fast.ai course for the practical. Or just the latter course, if you want to get right to it.

ML is one of the easiest fields out there. When I learned it I was actually turned off by how simplistic the concept was. Of course let me preface to say that it's hard to develop the intuition and skill in the same way learning to skateboard is hard. But conceptually it's easy and very possible for almost anyone.

The whole thing is just curve fitting. Literally finding some best fit curve across a series of points. This is very very easy for any software engineer to understand. I literally lost interest when I found out that the entire field was just all about messing with the data and the curve to try to get things to fit.

Literally it's just about eyeballing the data and qualitatively picking and training the thing that looks like it's the best fit. But because the data is N-dimensional and in the millions it's impossible to "eye-ball" it with your physical eyes, you have to come up with other techniques equivalent to "eye-balling" it.

Douglas Hofstadter had this whole theory of consciousness and when he found out that an LLM was a simple feed forward network with no feedback loops he went into a crisis. Basically his whole theory in GEB was wrong, according to him.

This stuff is NOT quantum physics. It's startling how simple it is and that's one of the big mysteries about it.

We only understand and build these things at a high level. At the very low level we don't actually understand what's going on. As I stated earlier we understand ML the same way a person understand data from an "eye-ball" perspective so it's impossible to even justify what exactly specifically went on with chatGPT when he answered a specific question correctly.