Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Low responsiveness of ML models to critical or deteriorating health conditions

nature.com

Post reply on HN
Print view View original

31–40 of 57 posts

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#31

Post by computerex »

klodolph wrote: post #26
wrsh07 wrote: post #22

Earlier quoted context omitted.

Linear regression is literally the second lecture of the Stanford ML class. https://cs229.stanford.edu/ If you want to say "not neural networks" or not dnn or not llm, sure. But it's obviously machine learning

When you say it’s “obviously machine learning”, how could that statement possibly be correct? There’s not even broad consensus here… so you don’t get to say that your definition is obviously correct. There are pedagogical reasons why you’d include linear regression in a machine learning course. This is pretty clear to me—they have properties which are extremely important to the field of machine learning field, such a…

Linear regression is ML. You are off base.

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#32

Post by Al-Khwarizmi »

rakejake wrote: post #27
magicalhippo wrote: post #5

For IHM prediction, LSTM models and transformer models were trained for 100 epochs using the MIMIC-III and eICU datasets separately. I might be blind, but I don't see any mention of loss. Did they stop at 100 because it was a nice round number or because it was a good place to stop? The LSTM model they used had 7k trainable parameters, the CW-LSTM model 153k while the transformer model had 800k parameters (300k train…

A 7k param LSTM is very tiny. Not sure if LSTMs would even work at that scale although someone with more theoretical knowledge can correct me on this. As an aside, I'm trying to train transformers for some classification tasks on audio data. The models are "small" (like 1M-15M params at most) and I find they are very finicky to train. Below 1M parameters I find them hard to train at all. I have thrown all sorts of le…

I don't have much help to offer, but just to echo your experience... at my group we have tried to train Transformers from scratch for various NLP tasks and we always have been hit with them being extremely brittle, and BiLSTMs working better. We only succeeded by following a pre-established recipe (e.g. training a BERT model from scratch for a new language, where the architecture, parameters and tasks are as in BERT), or of course by fine-tuning existing models, but just throwing some layers at a problem and training them from scratch... nope, won't work without arcane knowledge that doesn't seem to be written anywhere accessible. This is one of the reasons why I dislike Transformers and I root for the likes of RWKV to take the throne.

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#33

Post by genewitch »

computerex wrote: post #31
klodolph wrote: post #26

Earlier quoted context omitted.

When you say it’s “obviously machine learning”, how could that statement possibly be correct? There’s not even broad consensus here… so you don’t get to say that your definition is obviously correct. There are pedagogical reasons why you’d include linear regression in a machine learning course. This is pretty clear to me—they have properties which are extremely important to the field of machine learning field, such a…

Linear regression is ML. You are off base.

That's cool. Can you explain what machines they were using in the 1800s to do learning on?

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#35

Post by blackbear_ »

klodolph wrote: post #29
PaulHoule wrote: post #20

Earlier quoted context omitted.

I am sure there are people teach data science classes who look at it in that "reductive" way. From the viewpoint of engineering, scikit-learn provides the same interface to linear regression that it supplies to many other models. Huggingface provides an interface to models that is similar in a lot of ways but I think a 'regression' in that it doesn't provide the bare minimum of model selection facilities needed to re…

There’s a pedagogical reason to teach things with a kind of reductive definition. It makes a lot of sense. I remember getting cornered by somebody in a statistics class and interrogated about whether I thought neural networks were statistical techniques. In that situation I’ll only answer yes, they are statistical techniques. As far as I can tell, a big chunk of what we do with machine learning is create complicated…

> in traditional statistics, you can understand the parameters.

I also think that this is the key differentiator between ML and stats.

Statistical models can be understood formally, which means that not only we know how each parameter affects the predictions, we also know what their estimation uncertainties are, under which assumptions, and how to check that these assumptions are satisfied. Usually, we value these models not only because they're predictive but also because they're interpretable.

In ML there is neither the luxury nor the interest in doing this, all we want is something that predicts as well as possible.

So the difference is not the model itself but what you want to get out of it.

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#36

Post by timewizard »

All of this seems designed to make the hospital more labor efficient. None of this seems designed to improve long term outcomes for patients.

My continuing suspicion that this technology gets the hype it does as part of an effort to reduce wages for all workers grows.

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#37

Post by rakejake »

Al-Khwarizmi wrote: post #32
rakejake wrote: post #27

Earlier quoted context omitted.

A 7k param LSTM is very tiny. Not sure if LSTMs would even work at that scale although someone with more theoretical knowledge can correct me on this. As an aside, I'm trying to train transformers for some classification tasks on audio data. The models are "small" (like 1M-15M params at most) and I find they are very finicky to train. Below 1M parameters I find them hard to train at all. I have thrown all sorts of le…

I don't have much help to offer, but just to echo your experience... at my group we have tried to train Transformers from scratch for various NLP tasks and we always have been hit with them being extremely brittle, and BiLSTMs working better. We only succeeded by following a pre-established recipe (e.g. training a BERT model from scratch for a new language, where the architecture, parameters and tasks are as in BERT)…

I think the "arcane knowledge" is true for LLMs (billions). But there are lots of people who train models in the open in the hundreds of millions realm, but never below. Maybe transformers simply don't work as well below a size and data threshold.

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#38

Post by wswope »

AlotOfReading wrote: post #8
wswope wrote: post #3

I work in the ICU monitoring field, on the R&D team of a company with live systems at dozens of hospitals and multiple FDA approvals. We use extended Kalman filters (i.e. non-blackbox "ML") to estimate certain lab values of patients that are highly indicative of them crashing, based on live data from whatever set of monitors they're hooked up to - and it's highly robust. What the authors of this paper are doing is th…

Calling EKFs "ML" is certainly a choice.

Hence the quotes ;).

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#39

Post by getnormality »

AlotOfReading wrote: post #8
wswope wrote: post #3

I work in the ICU monitoring field, on the R&D team of a company with live systems at dozens of hospitals and multiple FDA approvals. We use extended Kalman filters (i.e. non-blackbox "ML") to estimate certain lab values of patients that are highly indicative of them crashing, based on live data from whatever set of monitors they're hooked up to - and it's highly robust. What the authors of this paper are doing is th…

Calling EKFs "ML" is certainly a choice.

It is a reasonable choice, and especially with the quotes around it, completely understandable.

The distinction between statistical inference and machine learning is too blurry to police Kalman filters onto one side.

Re: Low responsiveness of ML models to critical or deteriorating health conditions

#40

Post by Retric »

klodolph wrote: post #26
wrsh07 wrote: post #22

Earlier quoted context omitted.

Linear regression is literally the second lecture of the Stanford ML class. https://cs229.stanford.edu/ If you want to say "not neural networks" or not dnn or not llm, sure. But it's obviously machine learning

When you say it’s “obviously machine learning”, how could that statement possibly be correct? There’s not even broad consensus here… so you don’t get to say that your definition is obviously correct. There are pedagogical reasons why you’d include linear regression in a machine learning course. This is pretty clear to me—they have properties which are extremely important to the field of machine learning field, such a…

If you want to draw a line you could say anything that’s not full AGI isn’t machine learning because of philosophical reasons.

But, other than that there’s the only clear line is when the programmer isn’t hard coding results which puts Linear regression over the ML line. I guess you could argue about supervised vs unsupervised algorithms, but that’s going to exclude a lot of what is generally described as ML.

Post reply on HN