11–20 of 179 posts
Seems to completely ignore usage of local/free models as well as anything but Sonnet/ChatGPT. So my confidence in the good faith of the author is... heavily restricted.
I think this is somewhat disingenuous since not everyone uses the latest thing, and people tend to stick to “what works” for them. Models are picky enough about prompting styles that changing to a new model every week/month becomes an added chunk of cognitive overload, testing and experimentation, plus even in developer tooling there have been minor grating changes in API invocations and use of parameters like temper…
> I think this is somewhat disingenuous since not everyone uses the latest thing, and people tend to stick to “what works” for them. They say that the number of users on Claude 4.5 spiked and then a significant number of users reverted to 4.0 with the trend going up, and they are talking about their usage metrics. So I don't get how your comment is relevant to the article ?
Seems to completely ignore usage of local/free models as well as anything but Sonnet/ChatGPT. So my confidence in the good faith of the author is... heavily restricted.
Most people can’t affort the GPUs for local models if you want to get close to cloud capabilities.
GPT5 is HELLISHLY slow. That's all there is to it. It loves doing a whole bunch of reasoning steps and prolaim how mucf of a very good job it did clearing up its own todo steps and all that mumbo jumbo, but at the end of the day, I only asked it a small piece of information about nginx try_files that even GPT3 could answer instantly. Maybe before you make reasoning models that go on funny little sidequests wher they…
(§) You know that it's a hyperlink, do you? /s
Isn’t this obvious? When you have a task you think is hard. You give it to a cleverer model. When a task is straight forward you give it to an older one.
What is unclear from the presentation is wether they do this or not. Do teams that use Sonnet 4.5 just always use it, and teams on Sonnet 4.0 likewise? Or do individuals decided which model to use on a per task basis.
Personally I tend to default to just 1, and only go to an alternative if it gets stuck or doesn't get me what I want.
Nevertheless, 7 datapoints does not a trend make (and the data presented certainly doesnt explain why). The daily variation is more than I would have expected, but could also be down to what day of the week the pizza party is or the weekly scrum meetings is at a few of their customers workplaces.
We tried GPT-5 for a RAG use case, and found that it performs worse than 4.1. We reverted and didn't look back.
We tried GPT-5 for a RAG use case, and found that it performs worse than 4.1. We reverted and didn't look back.