i thought cursor was getting really bad, then i found out i was on a gpt 5 trial. gonna stick with claude :)
From GPT-4 to GPT-5: Measuring progress through MedHELM [pdf]
61–70 of 102 posts
Re: From GPT-4 to GPT-5: Measuring progress through MedHELM [pdf]
#62Earlier quoted context omitted.
> I get that. But then if that option doesn't help, what I've seen is that the next followup is inevitably "have you tried doing/prompting x instead of y" Maybe I’m misunderstanding, but it sounds like you’re framing a completely normal proces (try, fail, adjust) as if it’s unreasonable? In reality, when something doesn’t work, it would seem to me that the obvious next step is to adapt and try again. This does not se…
I get trying and improving until you get it right. But I just can't make the bridge in my head around 1. this is magic and will one-shot your questions 2. but if it goes wrong, keep trying until it works Plus, knowing it's all probabilistic, how do you know, without knowing ahead of time already, that the result is correct? Is that not the classic halting problem?
> I get trying and improving until you get it right. But I just can't make the bridge in my head around
> 1. this is magic and will one-shot your questions 2. but if it goes wrong, keep trying until it works
Ah that makes sense. I forgot the "magic" part, and was looking at it more practically.
Re: From GPT-4 to GPT-5: Measuring progress through MedHELM [pdf]
#63Earlier quoted context omitted.
True, but the experiments are engineered to give results they want. It's a mathematical certainty that the performance will drop off here, but is not an accurate assessment of what is going on at scale. If you present an appropriately large and well-trained model with in-context patterns, it often does a decent job, even when it isn't trained on them. By nerfing the model (4 layers), the conclusion is foregone. I hon…
Without a provable hold out, claim that "large models do fine on unseen patterns" is unfalsifiable. In controlled from scratch training, CoT performance collapses under modest distribution shift, even with plausible chains. If you have results where the transformation family is provably excluded from training and a large model still shows robust CoT, please share them. Otherwise this paper’s claim stands for the regi…
This paper's claim holds - for 4 layer models. Models improve on out of context examples dramatically at larger scales.
Re: From GPT-4 to GPT-5: Measuring progress through MedHELM [pdf]
#64Earlier quoted context omitted.
True, but the experiments are engineered to give results they want. It's a mathematical certainty that the performance will drop off here, but is not an accurate assessment of what is going on at scale. If you present an appropriately large and well-trained model with in-context patterns, it often does a decent job, even when it isn't trained on them. By nerfing the model (4 layers), the conclusion is foregone. I hon…
Without a provable hold out, claim that "large models do fine on unseen patterns" is unfalsifiable. In controlled from scratch training, CoT performance collapses under modest distribution shift, even with plausible chains. If you have results where the transformation family is provably excluded from training and a large model still shows robust CoT, please share them. Otherwise this paper’s claim stands for the regi…
> claim that "large models do fine on unseen patterns" is unfalsifiable
I know what you're saying here, and I know it is primarily a critique of my phrasing, but establishing something like this is the objective of in-context learning theory and mathematical applications of deep learning. It is possible to prove that sufficiently well-trained models will generalize for certain unseen classes of patterns, e.g. transformer acting like gradient descent. There is still a long way to go in the theory---it is difficult research!
> performance collapses under modest distribution shift
The problem is that the notion of "modest" depends on the scale here. With enough varied data and/or enough parameters, what was once out-of-distribution can become in-distribution. The paper is purposely ignorant of this fact. Yes, the claims hold for tiny models, but I don't think anyone ever doubted this.
Re: From GPT-4 to GPT-5: Measuring progress through MedHELM [pdf]
#65Feels like a mixed bag vs regression? eg - GPT-5 beats GPT-4 on factual recall + reasoning (HeadQA, Medbullets, MedCalc). But then slips on structured queries (EHRSQL), fairness (RaceBias), evidence QA (PubMedQA). Hallucination resistance better but only modestly. Latency seems uneven (maybe more testing?) faster on long tasks, slower on short ones.
GPT-5 feels like cost engineering. The model is incrementally better, but they are optimizing for least amount of compute. I am guessing investors love that.
I’ve found that it’s super likely to get stuck repeating the exact same incorrect response over and over. It used to happen occasionally with older models, but it happens frequently now.
Things like:
Me: Is this thing you claim documented? Where in the documentation does it say this?
GPT: Here’s a long-winded assertion that what I said before was correct, plus a link to an unofficial source that doesn’t back me up.
Me: That’s not official documentation and it doesn’t say what you claim. Find me the official word on the matter.
GPT: Exact same response, word-for-word.
Me: You are repeating yourself. Do not repeat what you said before. Here’s the official documentation: [link]. Find me the part where it says this. Do not consider any other source.
GPT: Exact same response, word-for-word.
Me: Here are some random words to test if you are listening to me: foo, bar, baz.
GPT: Exact same response, word-for-word.
It’s so repetitive I wonder if it’s an engineering fault, because it’s weird that the model would be so consistent in its responses regardless of the input. Once it gets stuck, it doesn’t matter what I enter, it just keeps saying the same thing over and over.
Re: From GPT-4 to GPT-5: Measuring progress through MedHELM [pdf]
#66Earlier quoted context omitted.
GPT-5 feels like cost engineering. The model is incrementally better, but they are optimizing for least amount of compute. I am guessing investors love that.
I’ve found that it’s super likely to get stuck repeating the exact same incorrect response over and over. It used to happen occasionally with older models, but it happens frequently now. Things like: Me: Is this thing you claim documented? Where in the documentation does it say this? GPT: Here’s a long-winded assertion that what I said before was correct, plus a link to an unofficial source that doesn’t back me up. M…
Go back and edit a prompt of yours in the conversation instead of continuing with garbage in the context.
Re: From GPT-4 to GPT-5: Measuring progress through MedHELM [pdf]
#67Earlier quoted context omitted.
I get trying and improving until you get it right. But I just can't make the bridge in my head around 1. this is magic and will one-shot your questions 2. but if it goes wrong, keep trying until it works Plus, knowing it's all probabilistic, how do you know, without knowing ahead of time already, that the result is correct? Is that not the classic halting problem?
> I get trying and improving until you get it right. But I just can't make the bridge in my head around > 1. this is magic and will one-shot your questions 2. but if it goes wrong, keep trying until it works Ah that makes sense. I forgot the "magic" part, and was looking at it more practically.
To clarify on the “learn and improve” part, I mean I get it in the context of a human doing it. When a person learns, that lesson sticks so errors and retries are valuable.
For LLMs none of it sticks. You keep “teaching” it and the next time it forgets everything.
So again you keep trying until you get the results you want, which you need to know ahead of time.
Re: From GPT-4 to GPT-5: Measuring progress through MedHELM [pdf]
#68Earlier quoted context omitted.
GPT-5 feels like cost engineering. The model is incrementally better, but they are optimizing for least amount of compute. I am guessing investors love that.
I agree. I have found GPT-5 significantly worse on medical queries. It feels like it skips important details and is much worse than o3, IMHO. I have heard good things about GPT-5 Pro, but that's not cheap. I wonder if part of the degraded performance is where they think you're going into a dangerous area and they get more and more vague, for example like they demoed on launch day with the fireworks example. It gets v…
Interesting, it seems the anecdotal experience agrees with the benchmark results.
Re: From GPT-4 to GPT-5: Measuring progress through MedHELM [pdf]
#69Feels like a mixed bag vs regression? eg - GPT-5 beats GPT-4 on factual recall + reasoning (HeadQA, Medbullets, MedCalc). But then slips on structured queries (EHRSQL), fairness (RaceBias), evidence QA (PubMedQA). Hallucination resistance better but only modestly. Latency seems uneven (maybe more testing?) faster on long tasks, slower on short ones.
Mixed results indeed. While it leads the benchmark in two question types, it falls short in others which results in the overall slight regression.
Re: From GPT-4 to GPT-5: Measuring progress through MedHELM [pdf]
#70Have you looked at comparing to Google's foundation models or specialty medical models like MedGemma ( https://developers.google.com/health-ai-developer-foundation... )?
That would be an interesting extension. MedGemma isn't part of the original benchmark either [1]. Since Gemini 2.0 Flash is on 6th place, expectations are for MedGemma to achieve higher than that :)
[1]https://crfm.stanford.edu/helm/medhelm/latest/#/leaderboard