Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4.1 in the API

openai.com

Post reply on HN
Print view View original

201–210 of 513 posts

Re: GPT-4.1 in the API

#201

Post by cootsnuck »

Tiberium wrote: post #88
minimaxir wrote: post #57

Earlier quoted context omitted.

OpenAI (and most LLM providers) allow model version pinning for exactly this reason, e.g. in the case of GPT-4o you can specify gpt-4o-2024-05-13, gpt-4o-2024-08-06, or gpt-4o-2024-11-20. https://platform.openai.com/docs/models/gpt-4o

Yes, and they don't make snapshots for chatgpt-4o-latest, but they made them for GPT 4.1, that's why 4.1 is only useful for API, since their ChatGPT product already has the better model.

Okay so is GPT 4.1 literally just the current chatpt-4o-latest or not?

Re: GPT-4.1 in the API

#202

Post by pzo »

impure wrote: post #93

I like how Nano matches Gemini 2.0 Flash's price. That will help drive down prices which will be good for my app. However I don't like how Nano behaves worse than 4o Mini in some benchmarks. Maybe it will be good enough, we'll see.

yeah and considering that gemini 2.0 flash is much better than 4o-mini. On top of that gemini have also audio input as modality and realtime API for both audio input and output + web search grounding + free tier.

Re: GPT-4.1 in the API

#203

Post by swyx »

don't miss that OAI also published a prompting guide WITH RECEIPTS for GPT 4.1 specifically for those building agents... with a new recommendation for:

- telling the model to be persistent (+20%)

- dont self-inject/parse toolcalls (+2%)

- prompted planning (+4%)

- JSON BAD - use XML or arxiv 2406.13121 (GDM format)

- put instructions + user query at TOP -and- BOTTOM - bottom-only is VERY BAD

- no evidence that ALL CAPS or Bribes or Tips or threats to grandma work

source: https://cookbook.openai.com/examples/gpt4-1_prompting_guide#...

Re: GPT-4.1 in the API

#204

Post by frognumber »

Marginally on-topic: I'd love if the charts included prior models, including GPT 4 and 3.5.

Not all systems upgrade every few months. A major question is when we reach step-improvements in performance warranting a re-eval, redesign of prompts, etc.

There's a small bleeding edge, and a much larger number of followers.

Re: GPT-4.1 in the API

#205

Post by swyx »

jsnell wrote: post #187
marsh_mellow wrote: post #142

From OpenAI's announcement: > Qodo tested GPT‑4.1 head-to-head against Claude Sonnet 3.7 on generating high-quality code reviews from GitHub pull requests. Across 200 real-world pull requests with the same prompts and conditions, they found that GPT‑4.1 produced the better suggestion in 55% of cases. Notably, they found that GPT‑4.1 excels at both precision (knowing when not to make suggestions) and comprehensiveness…

That's not a lot of samples for such a small effect, I don't think it's statistically significant (p-value of around 10%).

is there a shorthand/heuristic to calculate pvalue given n samples and effect size?

Re: GPT-4.1 in the API

#206

Post by swyx »

poormathskills wrote: post #70
ZeroCool2u wrote: post #8

No benchmark comparisons to other models, especially Gemini 2.5 Pro, is telling.

Go look at their past blog posts. OpenAI only ever benchmarks against their own models. This is pretty common across industries. The leader doesn’t compare themselves to the competition.

also sometimes if you get it wrong you catch unnecessary flak

Re: GPT-4.1 in the API

#207

Post by [deleted] »

frognumber wrote: post #204

Marginally on-topic: I'd love if the charts included prior models, including GPT 4 and 3.5. Not all systems upgrade every few months. A major question is when we reach step-improvements in performance warranting a re-eval, redesign of prompts, etc. There's a small bleeding edge, and a much larger number of followers.

[deleted]

Re: GPT-4.1 in the API

#208

Post by pzo »

glenstein wrote: post #69
meetpateltech wrote: post #23

GPT-4.1 Pricing (per 1M tokens): gpt-4.1 - Input: $2.00 - Cached Input: $0.50 - Output: $8.00 gpt-4.1-mini - Input: $0.40 - Cached Input: $0.10 - Output: $1.60 gpt-4.1-nano - Input: $0.10 - Cached Input: $0.025 - Output: $0.40

Awesome, thank you for posting. As someone who regularly uses 4o mini from the API, any guesses or intuitions about the performance of Nano? I'm not as concerned about nomenclature as other people, which I think is too often reacting to a headline as opposed to the article. But in this case, I'm not sure if I'm supposed to understand nano as categorically different than many in terms of what it means as a variation f…

they share in livestream that 4.1-nano is worse than 4o-mini - so nano is cheaper, faster and have bigger context but worse in intelligence. 4.1mini is smarter but there is price increase.

Re: GPT-4.1 in the API

#209

Post by bartkappenburg »

By leaving out scale or prior models they are effectively manipulating improvement. If from 3 to 4 it was from 10 to 80, and from 4 to 4o it was 80 to 82, leaving out 3 would let us see a steep line instead of steep decrease of growth.

Lies, damn lies and statistics ;-)

Post reply on HN