Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4.1 in the API

openai.com

Post reply on HN
Print view View original

171–180 of 513 posts

Re: GPT-4.1 in the API

#171

Post by camdenreslink »

Philpax wrote: post #121
camdenreslink wrote: post #87

Earlier quoted context omitted.

I have a medium-sized typescript personal project I work on. It probably has 20k LOC of well organized typescript (react frontend, express backend). I also have somewhat comprehensive docs and cursor project rules. In general I use Cursor in manual mode asking it to make very well scoped small changes (e.g. “write this function that does this in this exact spot”). Yesterday I needed to make a largely mechanical chang…

> Cursor agent mode puked all over itself using Gemini 2.5. It could summarize what changes would need to be made, but it was totally incapable of making the changes. Gemini 2.5 is currently broken with the Cursor agent; it doesn't seem to be able to issue tool calls correctly. I've been using Gemini to write plans, which Claude then executes, and this seems to work well as a workaround. Still unfortunate that it's l…

Interesting, I’ve found Gemini better than Claude so I defaulted to that. I’ll try another refactor in agent mode with Claude.

Re: GPT-4.1 in the API

#172

Post by egeozcan »

dmd wrote: post #17
ZeroCool2u wrote: post #8

No benchmark comparisons to other models, especially Gemini 2.5 Pro, is telling.

Gemini 2.5 Pro gets 64% on SWE-bench verified. Sonnet 3.7 gets 70% They are reporting that GPT-4.1 gets 55%.

Very interesting. For my use cases, Gemini's responses beat Sonnet 3.7's like 80% of the time (gut feeling, didn't collect actual data). It beats Sonnet 100% of the time when the context gets above 120k.

Re: GPT-4.1 in the API

#173

Post by Tinkeringz »

atemerev wrote: post #90
theturtletalks wrote: post #81

With these being 1M context size, does that all but confirm that Quasar Alpha and Optimus Alpha were cloaked OpenAI models on OpenRouter?

Yes, confirmed by citing Aider benchmarks: https://openai.com/index/gpt-4-1/ Which means that these models are _absolutely_ not SOTA, and Gemini 2.5 pro is much better, and Sonnet is better, and even R1 is better. Sorry Sam, you are losing the game.

Aren’t all of these reasoning models?

Won’t the reasoning models of openAI benchmarked against these be a test of if Sam is losing?

Re: GPT-4.1 in the API

#174

Post by marsh_mellow »

InkCanon wrote: post #149
marsh_mellow wrote: post #142

From OpenAI's announcement: > Qodo tested GPT‑4.1 head-to-head against Claude Sonnet 3.7 on generating high-quality code reviews from GitHub pull requests. Across 200 real-world pull requests with the same prompts and conditions, they found that GPT‑4.1 produced the better suggestion in 55% of cases. Notably, they found that GPT‑4.1 excels at both precision (knowing when not to make suggestions) and comprehensiveness…

>4.1 Was better in 55% of cases Um, isn't that just a fancy way of saying it is slightly better >Score of 6.81 against 6.66 So very slightly better

I don't think the absolute score means much — judge models have a tendency to score around 7/10 lol

55% vs. 45% equates to about a 36 point difference in ELO. in chess that would be two players in the same league but one with a clear edge

Re: GPT-4.1 in the API

#175

Post by pcwelder »

Did some quick tests. I believe its the same model as Quasar. It struggles with agentic loop [1]. You'd have to force it to do tool calls.

Tool use ability feels ability better than gemini-2.5-pro-exp [2] which struggles with JSON schema understanding sometimes.

Llama 4 has suprising agentic capabilities, better than both of them [3] but isn't as intelligent as the others.

[1] https://github.com/rusiaaman/chat.md/blob/main/samples/4.1/t...

[2] https://github.com/rusiaaman/chat.md/blob/main/samples/gemin...

[3] https://github.com/rusiaaman/chat.md/blob/main/samples/llama...

Re: GPT-4.1 in the API

#176

Post by hecticjeff »

chaos_emergent wrote: post #110
hmottestad wrote: post #31

Earlier quoted context omitted.

Are those with «thinking» or without?

based on their release cadence, I suspect that o4-mini will compete on price, performance, and context length with the rest of these models.

o4-mini, not to be confused with 4o-mini

Re: GPT-4.1 in the API

#177

Post by throwup238 »

handfuloflight wrote: post #155
coldcache wrote: post #151

Earlier quoted context omitted.

For reference, I think a common approximation is one token being 0.75 words. For a 100 page book, that translates to around 50,000 tokens. For 1 mil+ tokens, we need to be looking at 2000+ page books. That's pretty rare, even for documentation. It doesn't have to be text-based, though. I could see films and TV shows becoming increasingly important for long-context model training.

What about the role of synthetic data?

Synthetic data requires a discriminator that can select the highest quality results to feed back into training. Training a discriminator is easier than a full blown LLM, but it still suffers from a lack of high quality training data in the case of 1M context windows. How do you train a discriminator to select good 2,000 page synthetic books if the only ones you have to train it with are Proust and concatenated Harry Potter/Game of Thrones/etc.

Re: GPT-4.1 in the API

#178

Post by glenstein »

twistslider wrote: post #85
meetpateltech wrote: post #23

GPT-4.1 Pricing (per 1M tokens): gpt-4.1 - Input: $2.00 - Cached Input: $0.50 - Output: $8.00 gpt-4.1-mini - Input: $0.40 - Cached Input: $0.10 - Output: $1.60 gpt-4.1-nano - Input: $0.10 - Cached Input: $0.025 - Output: $0.40

The fact that they're raising the price for the mini models by 166% is pretty notable. gpt-4o-mini for comparison: - Input: $0.15 - Cached Input $0.075 - Output: $0.60

I don't think they ever committed themselves to uniformed pricing for mini models. Of course cheaper is better but I understand pricing to be contingent on factors specific to every next model rather than following from a blanket policy.

Re: GPT-4.1 in the API

#179

Post by zamadatix »

runako wrote: post #41

ChatGPT currently recommends I use o3-mini-high ("great at coding and logic") when I start a code conversation with 4o. I don't understand why the comparison in the announcement talks so much about comparing with 4o's coding abilities to 4.1. Wouldn't the relevant comparison be to o3-mini-high? 4.1 costs a lot more than o3-mini-high, so this seems like a pertinent thing for them to have addressed here. Maybe I am mis…

4.1 is a pinned API variant with the improvements from the newer iterations of 4o you're already using in the app, so that's why the comparison focuses between those two.

Pricing wise the per token cost of o3-mini is less than 4.1 but keep in mind o3-mini is a reasoning model and you will pay for those tokens too, not just the final output tokens. Also be aware reasoning models can take a long time to return a response... which isn't great if you're trying to use an API for interactive coding.

Post reply on HN