Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4.1 in the API

openai.com

Post reply on HN
Print view View original

281–290 of 513 posts

Re: GPT-4.1 in the API

#281

Post by chrisweekly »

wongarsu wrote: post #252
comex wrote: post #238

Sam Altman wrote in February that GPT-4.5 would be "our last non-chain-of-thought model" [1], but GPT-4.1 also does not have internal chain-of-thought [2]. It seems like OpenAI keeps changing its plans. Deprecating GPT-4.5 less than 2 months after introducing it also seems unlikely to be the original plan. Changing plans is necessarily a bad thing, but I wonder why. Did they not expect this model to turn out as well…

Maybe that's why they named this model 4.1, despite coming out after 4.5 and supposedly outperforming it. They can pretend GPT-4.5 is the last non-chain-of-thought model by just giving all non-chain-of-thought-models version numbers below 4.5

Ok, I know naming things is hard, but 4.1 comes out after 4.5? Just, wat.

Re: GPT-4.1 in the API

#282

Post by archeantus »

“GPT‑4.1 scores 54.6% on SWE-bench Verified, improving by 21.4%abs over GPT‑4o and 26.6%abs over GPT‑4.5—making it a leading model for coding.”

4.1 is 26.6% better at coding than 4.5. Got it. Also…see the em dash

Re: GPT-4.1 in the API

#283

Post by poormathskills »

christianqchung wrote: post #263
poormathskills wrote: post #70

Earlier quoted context omitted.

Go look at their past blog posts. OpenAI only ever benchmarks against their own models. This is pretty common across industries. The leader doesn’t compare themselves to the competition.

Okay, it's common across other industries, but not this one. Here is Google, Facebook, and Anthropic comparing their frontier models to others[1][2][3]. [1] https://blog.google/technology/google-deepmind/gemini-model-... [2] https://ai.meta.com/blog/llama-4-multimodal-intelligence/ [3] https://www.anthropic.com/claude/sonnet

Right. Those labs aren’t leading the industry.

Re: GPT-4.1 in the API

#284

Post by anotherpaulg »

jsnell wrote: post #163
modeless wrote: post #59

Numbers for SWE-bench Verified, Aider Polyglot, cost per million output tokens, output tokens per second, and knowledge cutoff month/year: SWE Aider Cost Fast Fresh Claude 3.7 70% 65% $15 77 8/24 Gemini 2.5 64% 69% $10 200 1/25 GPT-4.1 55% 53% $8 169 6/24 DeepSeek R1 49% 57% $2.2 22 7/24 Grok 3 Beta ? 53% $15 ? 11/24 I'm not sure this is really an apples-to-apples comparison as it may involve different test scaffoldi…

https://aider.chat/docs/leaderboards/ shows 73% rather than 69% for Gemini 2.5 Pro? Looks like they also added the cost of the benchmark run to the leaderboard, which is quite cool. Cost per output token is no longer representative of the actual cost when the number of tokens can vary by an order of magnitude for the same problem just based on how many thinking tokens the model is told to use.

Aider author here.

Based on some DMs with the Gemini team, they weren't aware that aider supports a "diff-fenced" edit format. And that it is specifically tuned to work well with Gemini models. So they didn't think to try it when they ran the aider benchmarks internally.

Beyond that, I spend significant energy tuning aider to work well with top models. That is in fact the entire reason for aider's benchmark suite: to quantitatively measure and improve how well aider works with LLMs.

Aider makes various adjustments to how it prompts and interacts with most every top model, to provide the very best possible AI coding results.

Re: GPT-4.1 in the API

#285

Post by spiderfarmer »

pbmango wrote: post #275

I think an under appreciated reality is that all of the large AI labs and OpenAI in particular are fighting multiple market battles at once. This is coming across in both the number of products and the packaging. 1, to win consumer growth they have continued to benefit on hyper viral moments, lately that was was image generation in 4o, which likely was technically possible a long time before launched. 2, for enterpri…

On that note, I want to see benchmarks for which LLM's are best at translating between languages. To me, it's an entire product category.

Re: GPT-4.1 in the API

#286

Post by intalentive »

Havoc wrote: post #253
swyx wrote: post #203

don't miss that OAI also published a prompting guide WITH RECEIPTS for GPT 4.1 specifically for those building agents... with a new recommendation for: - telling the model to be persistent (+20%) - dont self-inject/parse toolcalls (+2%) - prompted planning (+4%) - JSON BAD - use XML or arxiv 2406.13121 (GDM format) - put instructions + user query at TOP -and- BOTTOM - bottom-only is VERY BAD - no evidence that ALL CA…

>- dont self-inject/parse toolcalls (+2%) What is meant by this?

Use the OpenAI API/SDK for function calling instead of rolling your own inside the prompt.

Re: GPT-4.1 in the API

#287

Post by poormathskills »

awestroke wrote: post #280
poormathskills wrote: post #70

Earlier quoted context omitted.

Go look at their past blog posts. OpenAI only ever benchmarks against their own models. This is pretty common across industries. The leader doesn’t compare themselves to the competition.

Except they are far from the lead in model performance

Who has a (publicly released) model that is SOTA is constantly changing. It’s more interesting to see who is driving the innovation in the field, and right now that is pretty clearly OpenAI (GPT-3, first multi-modal model, first reasoning model, ect).

Re: GPT-4.1 in the API

#288

Post by sharkjacobs » 

    > You're eligible for free daily usage on traffic shared with OpenAI through April 30, 2025.
    > Up to 1 million tokens per day across gpt-4.5-preview, gpt-4.1, gpt-4o and o1
    > Up to 10 million tokens per day across gpt-4.1-mini, gpt-4.1-nano, gpt-4o-mini, o1-mini and o3-mini
    > Usage beyond these limits, as well as usage for other models, will be billed at standard rates. Some limitations apply.
I just found this option in https://platform.openai.com/settings/organization/data-contr...

Is just this something I haven't noticed before? Or is this new?

Re: GPT-4.1 in the API

#289

Post by zitterbewegung »

comex wrote: post #238

Sam Altman wrote in February that GPT-4.5 would be "our last non-chain-of-thought model" [1], but GPT-4.1 also does not have internal chain-of-thought [2]. It seems like OpenAI keeps changing its plans. Deprecating GPT-4.5 less than 2 months after introducing it also seems unlikely to be the original plan. Changing plans is necessarily a bad thing, but I wonder why. Did they not expect this model to turn out as well…

I think that people balked at the cost of 4.5 and really wanted just a slightly more improved 4o . Now it almost seems that they will have a separate products that are non chain of thought and chain of thought series which actually makes sense because some want a cheap model and some don't.

Re: GPT-4.1 in the API

#290

Post by sigmoid10 »

Cheer2171 wrote: post #258
comex wrote: post #238

Sam Altman wrote in February that GPT-4.5 would be "our last non-chain-of-thought model" [1], but GPT-4.1 also does not have internal chain-of-thought [2]. It seems like OpenAI keeps changing its plans. Deprecating GPT-4.5 less than 2 months after introducing it also seems unlikely to be the original plan. Changing plans is necessarily a bad thing, but I wonder why. Did they not expect this model to turn out as well…

Why do you expect to believe a single word Sam Altman says?

Everyone assumed malice when the board fired him for not always being "candid" - but it seems more and more that he's just clueless. He's definitely capable when it comes to raising money as a business, but I wouldn't count on any tech opinion from him.
Post reply on HN