GPT-4.1 in the API • Hacker News Board

Re: GPT-4.1 in the API

#401

Post by qingcharles » Tue, Apr 15, 2025, 3:13 AM UTC

throwup238 wrote: post #337

lxgr wrote: post #317

As a ChatGPT user, I'm weirdly happy that it's not available there yet. I already have to make a conscious choice between - 4o (can search the web, use Canvas, evaluate Python server-side, generate images, but has no chain of thought) - o3-mini (web search, CoT, canvas, but no image generation) - o1 (CoT, maybe better than o3, but no canvas or web search and also no images) - Deep Research (very powerful, but I have…

> - Deep Research (very powerful, but I have only 10 attempts per month, so I end up using roughly zero) Same here, which is a real shame. I've switched to DeepResearch with Gemini 2.5 Pro over the last few days where paid users have a 20/day limit instead of 10/month and it's been great, especially since now Gemini seems to browse 10x more pages than OpenAI Deep Research (on the order of 200-400 pages versus 20-40).…

Wow, I wondered what the limit was. I never checked, but I've been using it hesitantly since I burn up OpenAI's limit as soon as it resets. Thanks for the clarity.

I'm all-in on Deep Research. It can conduct research on niche historical topics that have no central articles in minutes, which typically were taking me days or weeks to delve into.

Re: GPT-4.1 in the API

#402

Post by girvo » Tue, Apr 15, 2025, 3:16 AM UTC

swyx wrote: post #362

girvo wrote: post #352

Earlier quoted context omitted.

> I don't see how it's any different than optimizing for new CPU/GPU architectures I mean that seems wild to say to me. Those architectures have documentation and aren't magic black boxes that we chuck inputs at and hope for the best: we do pretty much that with LLMs. If that's how you optimise, I'm genuinely shocked.

i bet if we talked to a real low level hardware systems/chip engineer they'd laugh and take another shot at how we put them on a pedestal

Not really, in my experience. There's still fundamental differences between designed systems and trained LLMs.

Re: GPT-4.1 in the API

#403

Post by anotherpaulg » Tue, Apr 15, 2025, 3:28 AM UTC

purplerabbit wrote: post #395

anotherpaulg wrote: post #370

Earlier quoted context omitted.

I just finished updating the aider polyglot leaderboard [0] with GPT-4.1, mini and nano. My results basically agree with OpenAI's published numbers. Results, with other models for comparison: Model Score Cost Gemini 2.5 Pro Preview 03-25 72.9% $ 6.32 claude-3-7-sonnet-20250219 64.9% $36.83 o3-mini (high) 60.4% $18.16 Grok 3 Beta 53.3% $11.03 * gpt-4.1 52.4% $ 9.86 Grok 3 Mini Beta (high) 49.3% $ 0.73 * gpt-4.1-mini 3…

What model are you personally using in your aider coding? :)

Mostly Gemini 2.5 Pro lately.

I get asked this often enough that I have a FAQ entry with automatically updating statistics [0].

  Model               Tokens     Pct

  Gemini 2.5 Pro   4,027,983   88.1%
  Sonnet 3.7         518,708   11.3%
  gpt-4.1-mini        11,775    0.3%
  gpt-4.1             10,687    0.2%

[0] https://aider.chat/docs/faq.html#what-llms-do-you-use-to-bui...

Re: GPT-4.1 in the API

#404

Post by taikahessu » Tue, Apr 15, 2025, 3:36 AM UTC

jonfw wrote: post #268

taikahessu wrote: post #234

Earlier quoted context omitted.

It has been really frustrating learning Godot (or any new technology you are not familiar with) 4.4.x with GPT4o or even worse, with custom GPT which use older GPT4turbo. As you are new in the field, it kinda doesn't make sense to pick an older version. It would be better if there was no data than incorrect data. You literally have to include the version number on every prompt and even that doesn't guarantee a right…

hey- curious what your experience has been like learning godot w/ LLM tooling. are you doing 3d? The 3D tutorial ecosystem is very GUI heavy and I have had major problems trying to get godot to do anything 3D

I'm afraid I'm only doing 2d ... Yes, GUI related LLM instructions have been exceptionally bad, with multiple prompts me saying "no there is no such thing"... But as I commented earlier, GPT has had it's moments.

I strongly recommend giving Gemini 2.5 Pro a shot. Personally I don't like their bloated UI, but you can set the temperature value, which is especially helpful when you are more certain what and how you want, then just lower that value. If you want to get some wilder ideas, turn it up. Also highly recommend reading the thought process it does! That was actually key in having very complex ideas working. Just spotting couple of lines there, that seem too vague or even just a little bit inaccurate ... then pasting them back, with your own comments, have helped me a ton.

Is there a specific part in which you struggle? And FWIW, I've been on a heavy learning spree for 2 weeks. I feel like I'm starting to see glimbses from the barrel's bottom ... it's not so deep, you just gotta hang in there and bombard different LLMs with different questions, different angles, stripping away most and trying the simplest variation, for both prompt and godot. Or sometimes by asking more general advice "what is current godot best practice in doing x".

And YouTube has also been helpful source, by listening how more experienced users make their stuff. You can mostly skim through the videos with doublespeed and just focus on how they are doing the basics. Best of luck!

Re: GPT-4.1 in the API

#405

Post by pzo » Tue, Apr 15, 2025, 3:55 AM UTC

anotherpaulg wrote: post #370

modeless wrote: post #59

Numbers for SWE-bench Verified, Aider Polyglot, cost per million output tokens, output tokens per second, and knowledge cutoff month/year: SWE Aider Cost Fast Fresh Claude 3.7 70% 65% $15 77 8/24 Gemini 2.5 64% 69% $10 200 1/25 GPT-4.1 55% 53% $8 169 6/24 DeepSeek R1 49% 57% $2.2 22 7/24 Grok 3 Beta ? 53% $15 ? 11/24 I'm not sure this is really an apples-to-apples comparison as it may involve different test scaffoldi…

I just finished updating the aider polyglot leaderboard [0] with GPT-4.1, mini and nano. My results basically agree with OpenAI's published numbers. Results, with other models for comparison: Model Score Cost Gemini 2.5 Pro Preview 03-25 72.9% $ 6.32 claude-3-7-sonnet-20250219 64.9% $36.83 o3-mini (high) 60.4% $18.16 Grok 3 Beta 53.3% $11.03 * gpt-4.1 52.4% $ 9.86 Grok 3 Mini Beta (high) 49.3% $ 0.73 * gpt-4.1-mini 3…

Did you benchmarked combo: DeepSeek R1 + DeepSeek V3 (0324)? There is combo on 3rd place : DeepSeek R1 + claude-3-5-sonnet-20241022 and also V3 new beating claude 3.5 so in theory R1 + V3 should be even on 2nd place. Just curious if that would be the case

Re: GPT-4.1 in the API

#406

Post by valenterry » Tue, Apr 15, 2025, 4:38 AM UTC

shortcord wrote: post #340

resters wrote: post #325

Earlier quoted context omitted.

I use them as follows: o1-pro: anything important involving accuracy or reasoning. Does the best at accomplishing things correctly in one go even with lots of context. deepseek R1: anything where I want high quality non-academic prose or poetry. Hands down the best model for these. Also very solid for fast and interesting analytical takes. I love bouncing ideas around with R1 and Grok-3 bc of their fast responses and…

Gemini 2.5 Pro is quite good at code. Has become my go to for use in Cursor. Claude 3.7 needs to be restrained too much.

Weird. For me, sonnet 3.7 is much more focussed and in particular works much better when finding the places that needs change and using other tooling. I guess the integration in cursor is just much better and more mature.

Re: GPT-4.1 in the API

#407

Post by ewoodrich » Tue, Apr 15, 2025, 4:39 AM UTC

staunton wrote: post #293

moojacob wrote: post #278

Earlier quoted context omitted.

> Things are going to continue getting better, faster, and weirder. I love this. Especially the weirder part. This tech can be useful in every crevice of society and we still have no idea what new creative use cases there are. Who would’ve guessed phones and social media would cause mass protests because bystanders could record and distribute videos of the police?

> Who would’ve guessed phones and social media would cause mass protests because bystanders could record and distribute videos of the police? That would have been quite far down on my list of "major (unexpected) consequences of phones and social media"...

Yep, it’s literally just a slightly higher tech version of (for example) the 1992 Los Angeles riots over Rodney King but with phones and Facebook instead of handheld camcorders and television.

Re: GPT-4.1 in the API

#408

Post by p1dda » Tue, Apr 15, 2025, 4:59 AM UTC

LLMs are not intelligent

Re: GPT-4.1 in the API

#409

Post by czk » Tue, Apr 15, 2025, 5:01 AM UTC

resters wrote: post #325

lxgr wrote: post #317

As a ChatGPT user, I'm weirdly happy that it's not available there yet. I already have to make a conscious choice between - 4o (can search the web, use Canvas, evaluate Python server-side, generate images, but has no chain of thought) - o3-mini (web search, CoT, canvas, but no image generation) - o1 (CoT, maybe better than o3, but no canvas or web search and also no images) - Deep Research (very powerful, but I have…

I use them as follows: o1-pro: anything important involving accuracy or reasoning. Does the best at accomplishing things correctly in one go even with lots of context. deepseek R1: anything where I want high quality non-academic prose or poetry. Hands down the best model for these. Also very solid for fast and interesting analytical takes. I love bouncing ideas around with R1 and Grok-3 bc of their fast responses and…

re: "grok-3 is r1 with mods" -- do you mean you believe they distilled deepseek r1? that was my assumption as well, though i thought it more jokingly at first it would make a lot of sense. i actually enjoy grok 3 quite a lot, it has some of the most entertaining thinking traces.

Re: GPT-4.1 in the API

#410

Post by krat0sprakhar » Tue, Apr 15, 2025, 5:05 AM UTC

simonw wrote: post #182

Here's a summary of this Hacker News thread created by GPT-4.1 (the full sized model) when the conversation hit 164 comments: https://gist.github.com/simonw/93b2a67a54667ac46a247e7c5a2fe... I think it did very well - it's clearly good at instruction following. Total token cost: 11,758 input, 2,743 output = 4.546 cents. Same experiment run with GPT-4.1 mini: https://gist.github.com/simonw/325e6e5e63d449cc5394e92b8f2a3…

Hey Simon, I love how you generates these summaries and share them on every model release. Do you have a quick script that allows you to do that? Would love to take a look if possible :)