Low responsiveness of ML models to critical or deteriorating health conditions • Hacker News Board

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#51

Post by jvanderbot » Sat, Mar 29, 2025, 11:39 PM UTC

PaulHoule wrote: post #15

thenobsta wrote: post #13

Earlier quoted context omitted.

Why not? LLMs, vision models, and kalman filters all learn parameters based on data.

A linear regression model can be written and trained as a neural net, has a loss function, all of that. Most if not all ML problems can be formulated as modelling a probability distribution

You've equated neural networks with ML.

I'm questioning the association of a single maximum likelihood parameter estimation via analytical optimization as ML, because it does not invole any methods beyond calculus and no models other than the system itself, whose parameters we are estimating.

Perhaps I'm wrong, but the power of NN is in an unknown intermediate representation between the data (measurements in estimation) and the prediction. EKF has no such black box.

Which now that I've written it agrees with top level comment so I rescind my question.

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#52

Post by bigmadshoe » Sun, Mar 30, 2025, 5:33 AM UTC

klodolph wrote: post #16

Epa095 wrote: post #11

Earlier quoted context omitted.

Is it less ML than linear regression?

If you want to draw the line between ML and not ML, I think you’ll have to put Kalman filters and linear regression on the non-ML side. You can put support vector machines and neural networks on the ML side. In some sense the exact place you draw the distinction is arbitrary. You could try to characterize where the distinction is by saying that models with fewer parameters and lower complexity tend to be called “not…

Linear regression is machine learning. At their core neural networks are just repeated linear regression + a non-linearity arranged in interesting ways. The key is that they can be trained to fit data using some optimization protocol (e.g. gradient descent). Just because linear regression has a closed form solution and is conceptually simple doesn't mean anything here.

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#53

Post by windsignaling » Sun, Mar 30, 2025, 10:10 AM UTC

jvanderbot wrote: post #50

windsignaling wrote: post #48

Earlier quoted context omitted.

Neural networks are not ML now?

EKF is a neural network!?

I think you missed the point of that comment. I was responding to the comment saying "Parameter estimation is ML now?"

Neural networks are trained commonly using maximum likelihood estimation, a common parameter estimation technique.

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#54

Post by zeroCalories » Sun, Mar 30, 2025, 6:48 PM UTC

wrsh07 wrote: post #41

genewitch wrote: post #33

Earlier quoted context omitted.

That's cool. Can you explain what machines they were using in the 1800s to do learning on?

Does this mean if I simulate a neural network on pen and paper that stops being machine learning? All of these are machine learning techniques. Doing it by hand doesn't change anything. Today we use machines so it's machine learning

I've worked out a 3x3 neural network with two pieces of labeled data by hand. Give me my statistics degree!

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#55

Post by zeroCalories » Sun, Mar 30, 2025, 7:01 PM UTC

klodolph wrote: post #18

PaulHoule wrote: post #15

Earlier quoted context omitted.

A linear regression model can be written and trained as a neural net, has a loss function, all of that. Most if not all ML problems can be formulated as modelling a probability distribution

That’s too reductive—ML models are statistical models. Statistical models have parameters, and in general cases, you choose the parameters with some kind of optimization algorithm. If you play fast and loose with your definition of “ML”, you’ll end up defining it so that any statistical model is an ML model… in which case, why even bother using two different terms? ML models are, broadly speaking, the more complicate…

I've heard professors call ML a form of applied statistics, and I think it's fair to call ML a subfield of statistics that deals with automatically generating statistical models with computers.

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#56

Post by [deleted] » Sun, Mar 30, 2025, 7:11 PM UTC

PaulHoule wrote: post #9

magicalhippo wrote: post #5

For IHM prediction, LSTM models and transformer models were trained for 100 epochs using the MIMIC-III and eICU datasets separately. I might be blind, but I don't see any mention of loss. Did they stop at 100 because it was a nice round number or because it was a good place to stop? The LSTM model they used had 7k trainable parameters, the CW-LSTM model 153k while the transformer model had 800k parameters (300k train…

What ever happened to early stopping? I see so many papers where people train neural networks with half-baked recipes. I think I saw early stopping first around 1990 but it is so often for people to pick some arbitrary number of epochs to run. I have to admit I never liked the term "early stopping", I think people should have called it just "stopping", because it makes it seem optional. Back when I was training LSTM…

[deleted]

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#57

Post by klodolph » Mon, Mar 31, 2025, 4:13 AM UTC

windsignaling wrote: post #46

klodolph wrote: post #26

Earlier quoted context omitted.

When you say it’s “obviously machine learning”, how could that statement possibly be correct? There’s not even broad consensus here… so you don’t get to say that your definition is obviously correct. There are pedagogical reasons why you’d include linear regression in a machine learning course. This is pretty clear to me—they have properties which are extremely important to the field of machine learning field, such a…

After spending over a decade in both statistics and machine learning I'd say the only reason there isn't a "broad consensus" is because statisticians like to gate-keep, whether that's linear regression, Monte Carlo methods, or Kalman Filters. Linear regression appears in pretty much every ML textbook. Can you confidently say, "this model that appears in every ML textbook is the only model in the ML textbook that isn'…

That’s unfairly harsh to statisticians, IMO. You have two fields of study, statistics and ML. There’s a massive overlap. Gatekeeping? Practitioners from these two fields have different jargon and view things differently from each other.

“X is taught in books about subject Y” is a pretty weak argument. I could use it to argue that group theory is quantum mechanics. Scientists and mathematics aren’t fighting over who gets to own group theory—the scientists get to put group theory in their toolboxes and the mathematicians get to study it for itself. Same with ML and statistics. When you do ML, you need certain statistical techniques in your toolbox, so they get taught in your ML books.