Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4.1 in the API

openai.com

Post reply on HN
Print view View original

351–360 of 513 posts

Re: GPT-4.1 in the API

#351

Post by bangaladore »

resters wrote: post #334
bangaladore wrote: post #329

Earlier quoted context omitted.

My experience is it often produces terrible diagrams. Things clearly overlap, lines make no sense. I'm not surprised as if you told me to layout a diagram in XML/YAML there would be obvious mistakes and layout issues. I'm not really certain a text output model can ever do well here.

FWIW I think a multimodal model could be trained to do extremely well with it given sufficient training data. A combination of textual description of the system and/or diagram, source code (mermaid, SVG, etc.) for the diagram, and the resulting image, with training to translate between all three.

Agreed. Even simply I'm sure a service like this already exists (or could easily exist) where the workflow is something like:

1. User provides information

2. LLM generates structured output for whatever modeling language

3. Same or other multimodal LLM reviews the generated graph for styling / positioning issues and ensure its matches user request.

4. LLM generates structured output based on the feedback.

5. etc...

But you could probably fine-tune a multimodal model to do it in one shot, or way more effectively.

Re: GPT-4.1 in the API

#352

Post by girvo »

RussianCow wrote: post #312
FridgeSeal wrote: post #306

Earlier quoted context omitted.

Usually when we’re doing it in practice there’s _somewhat_ more awareness of the mechanics than just throwing random obstructions in and hoping for the best.

LLMs are still very young. We'll get there in time. I don't see how it's any different than optimizing for new CPU/GPU architectures other than the fact that the latter is now a decades-old practice.

> I don't see how it's any different than optimizing for new CPU/GPU architectures

I mean that seems wild to say to me. Those architectures have documentation and aren't magic black boxes that we chuck inputs at and hope for the best: we do pretty much that with LLMs.

If that's how you optimise, I'm genuinely shocked.

Re: GPT-4.1 in the API

#353

Post by modeless »

anotherpaulg wrote: post #284
jsnell wrote: post #163

Earlier quoted context omitted.

https://aider.chat/docs/leaderboards/ shows 73% rather than 69% for Gemini 2.5 Pro? Looks like they also added the cost of the benchmark run to the leaderboard, which is quite cool. Cost per output token is no longer representative of the actual cost when the number of tokens can vary by an order of magnitude for the same problem just based on how many thinking tokens the model is told to use.

Aider author here. Based on some DMs with the Gemini team, they weren't aware that aider supports a "diff-fenced" edit format. And that it is specifically tuned to work well with Gemini models. So they didn't think to try it when they ran the aider benchmarks internally. Beyond that, I spend significant energy tuning aider to work well with top models. That is in fact the entire reason for aider's benchmark suite: to…

Thanks, that's interesting info. It seems to me that such tuning, while making Aider more useful, and making the benchmark useful in the specific context of deciding which model to use in Aider itself, reduces the value of the benchmark in evaluating overall model quality for use in other tools or contexts, as people use it for today. Models that get more tuning will outperform models that get less tuning, and existing models will have an advantage over new ones by virtue of already being tuned.

Re: GPT-4.1 in the API

#354

Post by cheschire »

cafeinux wrote: post #349
lxgr wrote: post #317

As a ChatGPT user, I'm weirdly happy that it's not available there yet. I already have to make a conscious choice between - 4o (can search the web, use Canvas, evaluate Python server-side, generate images, but has no chain of thought) - o3-mini (web search, CoT, canvas, but no image generation) - o1 (CoT, maybe better than o3, but no canvas or web search and also no images) - Deep Research (very powerful, but I have…

> 4.5 (better in creative writing, and probably warmer sound thanks to being vinyl based and using analog tube amplifiers, but slower and request limited, and I don't even know which of the other features it supports) Is that an LLM hallucination?

It’s a tongue in cheek reference to how audiophiles claim to hear differences in audio quality.

Re: GPT-4.1 in the API

#355

Post by SadTrombone »

cafeinux wrote: post #349
lxgr wrote: post #317

As a ChatGPT user, I'm weirdly happy that it's not available there yet. I already have to make a conscious choice between - 4o (can search the web, use Canvas, evaluate Python server-side, generate images, but has no chain of thought) - o3-mini (web search, CoT, canvas, but no image generation) - o1 (CoT, maybe better than o3, but no canvas or web search and also no images) - Deep Research (very powerful, but I have…

> 4.5 (better in creative writing, and probably warmer sound thanks to being vinyl based and using analog tube amplifiers, but slower and request limited, and I don't even know which of the other features it supports) Is that an LLM hallucination?

Pretty dark times on HN, when a silly (and obvious) joke gets someone labeled as AI.

Re: GPT-4.1 in the API

#356

Post by BonoboIO »

anotherpaulg wrote: post #284
jsnell wrote: post #163

Earlier quoted context omitted.

https://aider.chat/docs/leaderboards/ shows 73% rather than 69% for Gemini 2.5 Pro? Looks like they also added the cost of the benchmark run to the leaderboard, which is quite cool. Cost per output token is no longer representative of the actual cost when the number of tokens can vary by an order of magnitude for the same problem just based on how many thinking tokens the model is told to use.

Aider author here. Based on some DMs with the Gemini team, they weren't aware that aider supports a "diff-fenced" edit format. And that it is specifically tuned to work well with Gemini models. So they didn't think to try it when they ran the aider benchmarks internally. Beyond that, I spend significant energy tuning aider to work well with top models. That is in fact the entire reason for aider's benchmark suite: to…

Thank you for providing such amazing tools for us. Aider is a godsend, when working with large codebase to get an overview.

Re: GPT-4.1 in the API

#357

Post by rushingcreek »

resters wrote: post #325
lxgr wrote: post #317

As a ChatGPT user, I'm weirdly happy that it's not available there yet. I already have to make a conscious choice between - 4o (can search the web, use Canvas, evaluate Python server-side, generate images, but has no chain of thought) - o3-mini (web search, CoT, canvas, but no image generation) - o1 (CoT, maybe better than o3, but no canvas or web search and also no images) - Deep Research (very powerful, but I have…

I use them as follows: o1-pro: anything important involving accuracy or reasoning. Does the best at accomplishing things correctly in one go even with lots of context. deepseek R1: anything where I want high quality non-academic prose or poetry. Hands down the best model for these. Also very solid for fast and interesting analytical takes. I love bouncing ideas around with R1 and Grok-3 bc of their fast responses and…

Phind was fine-tuned specifically to produce inline Mermaid diagrams for technical questions (I'm the founder).

Re: GPT-4.1 in the API

#358

Post by bangaladore »

XCSme wrote: post #302
sharkjacobs wrote: post #288

> You're eligible for free daily usage on traffic shared with OpenAI through April 30, 2025. > Up to 1 million tokens per day across gpt-4.5-preview, gpt-4.1, gpt-4o and o1 > Up to 10 million tokens per day across gpt-4.1-mini, gpt-4.1-nano, gpt-4o-mini, o1-mini and o3-mini > Usage beyond these limits, as well as usage for other models, will be billed at standard rates. Some limitations apply. I just found this optio…

So, that's like $10/day to give all your data/prompts?

IIRC 4.5 was 75$/1M input and 150$/1M output.

O1 is 15$ in 60$ out.

So you could easily get 75+$ per day free from this.

Re: GPT-4.1 in the API

#359

Post by mianos »

999900000999 wrote: post #227

Have they implemented "I don't know" yet. I probably spend 100$ a month on AI coding, and it's great at small straightforward tasks. Drop it into a larger codebase and it'll get confused. Even if the same tool built it in the first place due to context limits. Then again, the way things are rapidly improving I suspect I can wait 6 months and they'll have a model that can do what I want.

I agree. I use it a lot but there is endless frustration when the C++ code I am working on gets both complex and largish. Once it gets to a certain size and the context gets too long they all pretty much lose the plot and start producing complete rubbish. It would be great for it to give some measure so I know to take over and not have it start injecting random bugs or deleting functional code. It even starts doing things like returning locally allocated pointers lately.

Re: GPT-4.1 in the API

#360

Post by osigurdson »

Sam made a strange statement imo in a recent Ted Talk. He said (something like) models come and go but they want to be the best platform.

For me, it was jaw dropping. Perhaps he didn't mean it the way it sounded, but seemed like a major shift to me.

Post reply on HN