Lots of improvements here (hopefully), but still no image generation updates, which is what I'm most eager for right now.
Or text to speech generation ... but I guess that is coming.
GPT-4.1 in the API
141–150 of 513 posts
Re: GPT-4.1 in the API
#142From OpenAI's announcement:
> Qodo tested GPT‑4.1 head-to-head against Claude Sonnet 3.7 on generating high-quality code reviews from GitHub pull requests. Across 200 real-world pull requests with the same prompts and conditions, they found that GPT‑4.1 produced the better suggestion in 55% of cases. Notably, they found that GPT‑4.1 excels at both precision (knowing when not to make suggestions) and comprehensiveness (providing thorough analysis when warranted).
Re: GPT-4.1 in the API
#143it's worse than 4.5 on nearly every benchmark. just an incremental improvement. AI is slowing down
Sorry what is the source for this?
Re: GPT-4.1 in the API
#144it's worse than 4.5 on nearly every benchmark. just an incremental improvement. AI is slowing down
They don't disclose parameter counts so it's hard to say exactly how far apart they are in terms of size, but based on the pricing it seems like a pretty wild comparison, with one being an attempt at an ultra-massive SOTA model and one being a model scaled down for efficiency and probably distilled from the big one. The way they're presented as version numbers is business nonsense which obscures a lot about what's going on.
Re: GPT-4.1 in the API
#145Could any one guess the reason as to why they didn't ship this in the chat UI?
Re: GPT-4.1 in the API
#146From OpenAI's announcement: > Qodo tested GPT‑4.1 head-to-head against Claude Sonnet 3.7 on generating high-quality code reviews from GitHub pull requests. Across 200 real-world pull requests with the same prompts and conditions, they found that GPT‑4.1 produced the better suggestion in 55% of cases. Notably, they found that GPT‑4.1 excels at both precision (knowing when not to make suggestions) and comprehensiveness…
Interesting link. Worth noting that the pull requests were judged by o3-mini. Further, I'm not sure that 55% vs 45% is a huge difference.
Re: GPT-4.1 in the API
#147> They feature a refreshed knowledge cutoff of June 2024. As opposed to Gemini 2.5 Pro having cutoff of Jan 2025. Honestly this feels underwhelming and surprising. Especially if you're coding with frameworks with breaking changes, this can hurt you.
It's definitely an issue. Even the simplest use case of "create React app with Vite and Tailwind" is broken with these models right now because they're not up to date.
usually enabling "Search" fixes it sometimes as they fetch the newer methods.
Re: GPT-4.1 in the API
#148Could any one guess the reason as to why they didn't ship this in the chat UI?
The memory thing? More resources intensive?
Re: GPT-4.1 in the API
#149From OpenAI's announcement: > Qodo tested GPT‑4.1 head-to-head against Claude Sonnet 3.7 on generating high-quality code reviews from GitHub pull requests. Across 200 real-world pull requests with the same prompts and conditions, they found that GPT‑4.1 produced the better suggestion in 55% of cases. Notably, they found that GPT‑4.1 excels at both precision (knowing when not to make suggestions) and comprehensiveness…
>4.1 Was better in 55% of cases
Um, isn't that just a fancy way of saying it is slightly better
>Score of 6.81 against 6.66
So very slightly better
Re: GPT-4.1 in the API
#150Lots of improvements here (hopefully), but still no image generation updates, which is what I'm most eager for right now.
Or text to speech generation ... but I guess that is coming.
Yeah, I tried the 4o models and they severely mispronounced common words and read numbers incorrectly (eg reading 16000 as 1600)