Linear Regression

I'd say arguing about this is a huge distraction. Some would say classification is a special case of regression: the predicted continuous values are interpreted as class probabilities.

You're right that regression is a type of task, not a type of solution. The conceptual difference is important to understand if someone does not yet know it. But it's not an all or nothing set-in-stone thing. It implies there is a waterfall design to these ideas, as if some oracle posed these tasks to us and then we started finding solutions to each of them separately. But actually in many cases one and the same algorithm with small tweaks can tackle multiple tasks. Sometimes we have the hammer (algorithm) first and then the nails (task).

How you build your taxonomy and how you categorize one approach or another is not the same as learning and understanding. I always had an issue with this at university, where some lecturers would confuse learning lists like "what are the 3 areas of field X" or what are "the 4 principles of approach Y" etc.

The world is not structured according to subjects, fields, subfields etc. It's not a single hierarchy, but a big mess of similarities, like a graph or a multidimensional space. The map is not the territory and so on.

Terminology is necessary of course for communication and structuring books etc, but I like to see it merely as a utilitarian thing. The categories provide a scaffolding so that learning can happen. Studying the vocabulary and various outlines and nested hierarchies is a useful part of the journey but should not be confused with actually learning the thing itself. You could in principle learn all about regression without ever learning the word "regression".

The name actually originates from statistics, named after the "regression towards the mean" phenomenon, where they observed how the adult height of people is closer to the average compared to the height of their parents. So a tall person will have tall children but less extremely tall (in tendency, some will of course be even taller). So confusingly, regression literally means "going back".

Why do people write or even read posts like this. You can crack open any stats textbook and it will be explained there in much more detail by someone likely far more qualified to be talking about it.

Feels like some mix of SEO-farming/resume-building-blogspam.

The discussion here suggests that a lot of developers are jumping right into the middle of data science from a machine learning perspective without a solid understanding of what I would call basic math. It's a lot easier to succeed and compete with a solid grasp of linear algebra, calculus and numerical methods. My personal experience is that curriculum takes a good couple years to really get your head around.

For my current project I had to really understand the closed-form solution for linear regression and even it looks straightforward on paper it really took a while to sink in.

The author wanted to use the door emoji.

Yes. In fact, an even better solution would be using QR factorization...