Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4.1 in the API

openai.com

Post reply on HN
Print view View original

151–160 of 513 posts

Re: GPT-4.1 in the API

#151

Post by coldcache »

roflmaostc wrote: post #130
kmeisthax wrote: post #117

Earlier quoted context omitted.

The problem is that while you can train a model with the hyperparameter of "context size" set to 1M, there's very little 1M data to train on. Most of your model's ability to follow long context comes from the fact that it's trained on lots of (stolen) books; in fact I believe OpenAI just outright said in court that they can't do long context without training on books. Novels are usually measured in terms of words; an…

What about old books? Wikipedia? Law texts? Programming languages documentations? How many tokens is a 100 pages PDF? 10k to 100k?

For reference, I think a common approximation is one token being 0.75 words.

For a 100 page book, that translates to around 50,000 tokens. For 1 mil+ tokens, we need to be looking at 2000+ page books. That's pretty rare, even for documentation.

It doesn't have to be text-based, though. I could see films and TV shows becoming increasingly important for long-context model training.

Re: GPT-4.1 in the API

#152

Post by marsh_mellow »

coldcache wrote: post #146
marsh_mellow wrote: post #142

From OpenAI's announcement: > Qodo tested GPT‑4.1 head-to-head against Claude Sonnet 3.7 on generating high-quality code reviews from GitHub pull requests. Across 200 real-world pull requests with the same prompts and conditions, they found that GPT‑4.1 produced the better suggestion in 55% of cases. Notably, they found that GPT‑4.1 excels at both precision (knowing when not to make suggestions) and comprehensiveness…

Interesting link. Worth noting that the pull requests were judged by o3-mini. Further, I'm not sure that 55% vs 45% is a huge difference.

Good point. They said they validated the results by testing with other models (including Claude), as well as with manual sanity checks.

55% to 45% definitely isn't a blowout but it is meaningful — in terms of ELO it equates to about a 36 point difference. So not in a different league but definitely a clear edge

Re: GPT-4.1 in the API

#153

Post by mrandish »

Philpax wrote: post #29
codingwagie wrote: post #10

GPT-4.1 probably is a distilled version of GPT-4.5 I dont understand the constant complaining about naming conventions. The number system differentiates the models based on capability, any other method would not do that. After ten models with random names like "gemini", "nebula" you would have no idea which is which. Its a low IQ take. You dont name new versions of software as completely different software Also, Yest…

> The number system differentiates the models based on capability, any other method would not do that. Please rank GPT-4, GPT-4 Turbo, GPT-4o, GPT-4.1-nano, GPT-4.1-mini, GPT-4.1, GPT-4.5, o1-mini, o1, o1 pro, o3-mini, o3-mini-high, o3, and o4-mini in terms of capability without consulting any documentation.

Yes, point taken.

However, it's still not as bad as Intel CPU naming in some generations or USB naming (until very recently). I know, that's a very low bar... :-)

Re: GPT-4.1 in the API

#154

Post by rfw300 »

exizt88 wrote: post #56
Tiberium wrote: post #42

Very important note: >Note that GPT‑4.1 will only be available via the API. In ChatGPT, many of the improvements in instruction following, coding, and intelligence have been gradually incorporated into the latest version If anyone here doesn't know, OpenAI does offer the ChatGPT model version in the API as chatgpt-4o-latest, but it's bad because they continuously update it so businesses can't reliably rely on it bein…

> chatgpt-4o-latest, but it's bad because they continuously update it Version explicitly marked as "latest" being continuously updated it? Crazy.

It can be both properly communicated and still bad for API use cases.

Re: GPT-4.1 in the API

#155

Post by handfuloflight »

coldcache wrote: post #151
roflmaostc wrote: post #130

Earlier quoted context omitted.

What about old books? Wikipedia? Law texts? Programming languages documentations? How many tokens is a 100 pages PDF? 10k to 100k?

For reference, I think a common approximation is one token being 0.75 words. For a 100 page book, that translates to around 50,000 tokens. For 1 mil+ tokens, we need to be looking at 2000+ page books. That's pretty rare, even for documentation. It doesn't have to be text-based, though. I could see films and TV shows becoming increasingly important for long-context model training.

What about the role of synthetic data?

Re: GPT-4.1 in the API

#156

Post by lukev »

forbiddenvoid wrote: post #129
taikahessu wrote: post #124

> They feature a refreshed knowledge cutoff of June 2024. As opposed to Gemini 2.5 Pro having cutoff of Jan 2025. Honestly this feels underwhelming and surprising. Especially if you're coding with frameworks with breaking changes, this can hurt you.

It's definitely an issue. Even the simplest use case of "create React app with Vite and Tailwind" is broken with these models right now because they're not up to date.

Time to start moving back to Java & Spring.

100% backwards compatibility and well represented in 15 years worth of training data, hah.

Re: GPT-4.1 in the API

#157

Post by kevmo314 »

InkCanon wrote: post #149
marsh_mellow wrote: post #142

From OpenAI's announcement: > Qodo tested GPT‑4.1 head-to-head against Claude Sonnet 3.7 on generating high-quality code reviews from GitHub pull requests. Across 200 real-world pull requests with the same prompts and conditions, they found that GPT‑4.1 produced the better suggestion in 55% of cases. Notably, they found that GPT‑4.1 excels at both precision (knowing when not to make suggestions) and comprehensiveness…

>4.1 Was better in 55% of cases Um, isn't that just a fancy way of saying it is slightly better >Score of 6.81 against 6.66 So very slightly better

A great way to upsell 2% better! I should start doing that.

Re: GPT-4.1 in the API

#158

Post by druskacik »

twistslider wrote: post #85
meetpateltech wrote: post #23

GPT-4.1 Pricing (per 1M tokens): gpt-4.1 - Input: $2.00 - Cached Input: $0.50 - Output: $8.00 gpt-4.1-mini - Input: $0.40 - Cached Input: $0.10 - Output: $1.60 gpt-4.1-nano - Input: $0.10 - Cached Input: $0.025 - Output: $0.40

The fact that they're raising the price for the mini models by 166% is pretty notable. gpt-4o-mini for comparison: - Input: $0.15 - Cached Input $0.075 - Output: $0.60

That's what I was thinking. I hoped to see a price drop, but this does not change anything for my use cases.

I was using gpt-4o-mini with batch API, which I recently replaced with mistral-small-latest batch API, which costs $0.10/$0.30 (or $0.05/$0.15 when using the batch API). I may change to 4.1-nano, but I'd have to be overwhelmed by its performance in comparision to mistral.

Re: GPT-4.1 in the API

#160

Post by tomjen3 »

modeless wrote: post #59

Numbers for SWE-bench Verified, Aider Polyglot, cost per million output tokens, output tokens per second, and knowledge cutoff month/year: SWE Aider Cost Fast Fresh Claude 3.7 70% 65% $15 77 8/24 Gemini 2.5 64% 69% $10 200 1/25 GPT-4.1 55% 53% $8 169 6/24 DeepSeek R1 49% 57% $2.2 22 7/24 Grok 3 Beta ? 53% $15 ? 11/24 I'm not sure this is really an apples-to-apples comparison as it may involve different test scaffoldi…

Its available for free in Windsurf so you can try it out there.

Edit: Now also in Cursor

Post reply on HN