Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advancing the price-performance frontier with GPT‑5.6

openai.com

Post reply on HN
Print view View original

251–260 of 424 posts

Re: Advancing the price-performance frontier with GPT‑5.6

#251

Post by oh_no »

pixelesque wrote: post #79
maxdo wrote: post #48

Earlier quoted context omitted.

is kimi that cheap? it's a very expensive model

It's cheaper currently on many of the inference providers. Personally, I'm having surprisingly good results with DeepSeek 4 Pro at home, which is very good value for money: it's not as good as Claude / GPT 5.6 (I have Co-pilot license at work), but it's still really useful for code reviews, validating thoughts, and especially designing / writing unit tests for new (and old before refactoring) functionality. And it's…

where are you seeing cheap Kimi? pricing I've seen is the same across the board (presumably due to licensing terms) and is in the Terra range.

Re: Advancing the price-performance frontier with GPT‑5.6

#252

Post by WalterGR »

ElijahLynn wrote: post #200
jaggederest wrote: post #131

Earlier quoted context omitted.

https://taalas.com/ has done it already for a wildly obsolete model. 14000 tokens per second. https://chatjimmy.ai/ is their interactive. Tiny context, very dumb, but absurdly fast. Imagine this as a tool call for claude code for trivial changes - the tool call from the harness takes longer than the execution.

Wow! You weren't kidding, I just tried it too and 14,098 tokens in .05 seconds, I barely blinked and it was done. There was no typing at all appearing on the screen. It just showed up. https://chatjimmy.ai/chats/01dc66a4-4b1b-4dea-bb5f-926855e37...

That link isn't bringing up your chat, FYI. It just shows the default new chat state.

Re: Advancing the price-performance frontier with GPT‑5.6

#253

Post by FuriouslyAdrift »

monkeydust wrote: post #244
FuriouslyAdrift wrote: post #214

Earlier quoted context omitted.

Yep it will be ASICs and DSPs all over again. Orders of magnitude changes.

So which shovels companies are the ones to watch for burnt in silicon models ?

Imagine the price of a $9 million NVL72 dropped to about $100, used 4 orders of magnitude less power, was the size of ARM cpu, be bundled with pretty much any electronic device, and ran as fast a frontier AI is today.

That's about how disrupting DSPs were to the industries they arose out of (over a very long time frame).

How would that disrupt the industry?

Re: Advancing the price-performance frontier with GPT‑5.6

#254

Post by christophilus »

jaggederest wrote: post #131
jrflo wrote: post #115

Earlier quoted context omitted.

Burning the weights into silicon would be many orders of magnitude increase, not just 10x. It's kind of crazy that this hockey stick the AI hype bros talk about seems more and more every day like it might be real

https://taalas.com/ has done it already for a wildly obsolete model. 14000 tokens per second. https://chatjimmy.ai/ is their interactive. Tiny context, very dumb, but absurdly fast. Imagine this as a tool call for claude code for trivial changes - the tool call from the harness takes longer than the execution.

Wow. This is absolutely wild. I didn't expect that.

If we get to anywhere near this speed for the equivalent of the current models... I don't even know what to think about that future.

Re: Advancing the price-performance frontier with GPT‑5.6

#255

Post by jmb99 »

kridsdale1 wrote: post #212
re-thc wrote: post #116

Earlier quoted context omitted.

> When model intelligence reliably hits 90%-95% of current day knowledge worker tasks, they are going to burn those weight to silicon Google is already working on a similar idea but more "flexible".

Explain.

I'm not who you responded to and I don't have any info on Google. Nor can I explain in detail due to NDAs. But multiple major players are working on something along the lines of what the parent is alluding to.

The "edge" AI landscape (in particular, what you can do with ~5W) is going to be nuts in about 18 months.

Re: Advancing the price-performance frontier with GPT‑5.6

#256

Post by archon1410 »

randomblock1 wrote: post #241
archon1410 wrote: post #211

If Luna is so good and cheap, I'm wondering why ChatGPT Free users still only have access to GPT 5.5 Instant. The API pricing for chat-latest is the same as Sol(!),[1] while Luna is 25× cheaper [2]. Even assuming highly inflated API pricing for chat-latest, Luna not being the daily driver on Free and Go plans feels off. Perhaps users prefer instant responses over thinking models so much so that using a more expensive…

I bet they'll switch over pretty soon. They always make free users use the older models for a little bit, probably to try to push people to upgrade. You can actually use Luna without reasoning (set it to "none"). So if they wanted to, they could definitely replace 5.5 Instant with it.

Good points. Though GPT 5.5 Instant does beat out 5.6 Luna non-thinking 34 to 27 on Artificial Analysis Intelligence Index.[1][2] This could mean it is optimised for non-reasoning perfomance. So even if beaten on cost and performance, it could still be on the pareto frontier for speed.

It probably costs them a substantial amount of money everyday to not replace Instant with Luna, and the people who want larger models will upgrade regardless of whether they get Instant or Luna on the free tier. It is unlikely the amount of people upgrading just to have latest models would be enough to offset the costs.

[1]https://artificialanalysis.ai/models/gpt-5-6-luna-non-reason... [2]https://artificialanalysis.ai/models/gpt-5-5-instant-05-26

Re: Advancing the price-performance frontier with GPT‑5.6

#257

Post by throwup238 »

8cvor6j844qw_d6 wrote: post #166
iamjackg wrote: post #137

Earlier quoted context omitted.

Holy crap, I was not prepared for how fast it responded. I just wrote "Just wanted to see how fast you are! Can you write me a quick story about a tiger who lives inside a block of cheese the size of a house?" I pressed Enter, and the response was instant . > Generated in 0.037s • 14,205 tok/s This is unbelievable.

I'd like to imagine the things that can be done with this speed and the current frontier models.

Fully interactive games where you can talk to every NPC by text or voice and have an LLM drive the story (with your own meta prompts to guide it, if you so wish). Maybe even have them generate assets on the fly too.

I’m still trying to figure out coding agents. I can’t even begin to imagine the things it would enable. Even the most mundane ideas like LLMs-in-HiFreq-trading have huge implications.

Re: Advancing the price-performance frontier with GPT‑5.6

#258

Post by NitpickLawyer »

magicalist wrote: post #243
NitpickLawyer wrote: post #17

Earlier quoted context omitted.

~2 years ago gemini2.5 helped write better kernes for itself and (only) reached 1% efficiency gains. Today we're at 20%.

If you optimize program A and manage to wring out a 1% improvement, and I optimize program B and improve performance by 20%, you can see the problem with trying to infer anything from those two numbers. Edit: searching for the story now, further bolstering the point is that was 1% in training time [1], and the openAI claim is 20% in end to end inference cost . This is a bad comparison. [1] https://deepmind.google/blo…

> This is a bad comparison.

How so? First, kernel writing (or ML engineering more broadly) is a highly specialised task. Not everyone can do it. It shows that models are getting better and better at (easily verifiable) hard tasks. And you can "hire" that expertise much easier than you can hire the equivalent meatbags. And more importantly you can "fire" them as soon as the task is done. And then hire them 3 months later, when the new model drops. And so on.

Second, 20% gains in inference today gives better end results (i.e. lower overall cost) than 1% in training 2 years ago. Today's models are improving mostly via RL. And RL is highly dependant on fast inference (you want many rollouts for each training scenario). Same for dataset filtering, environment generation, distillation, etc.

Re: Advancing the price-performance frontier with GPT‑5.6

#259

Post by vjvjvjvjghv »

jrflo wrote: post #115
jpadkins wrote: post #86

Earlier quoted context omitted.

When model intelligence reliably hits 90%-95% of current day knowledge worker tasks, they are going to burn those weight to silicon and we will see another 10X improvement in price/performance frontier. The dynamic GPU clusters will be used for the 5% of tasks, and pushing out the frontier. Also there will be a set of knowledge tasks that are not done today (because they are too difficult for most knowledge workers),…

Burning the weights into silicon would be many orders of magnitude increase, not just 10x. It's kind of crazy that this hockey stick the AI hype bros talk about seems more and more every day like it might be real

Not an expert on this but wouldn’t this be possible with something similar to an FPGA?

Re: Advancing the price-performance frontier with GPT‑5.6

#260

Post by forsalebypwner »

fy20 wrote: post #242
pixelesque wrote: post #79

Earlier quoted context omitted.

It's cheaper currently on many of the inference providers. Personally, I'm having surprisingly good results with DeepSeek 4 Pro at home, which is very good value for money: it's not as good as Claude / GPT 5.6 (I have Co-pilot license at work), but it's still really useful for code reviews, validating thoughts, and especially designing / writing unit tests for new (and old before refactoring) functionality. And it's…

DeepSeek V4 Pro is ridiculously priced, especially when you take into account caching. According to the DeepSeek usage panel, 50M tokens have cost me $1.38. It's not the smartest and does like to overthink, but if you have well defined problems it's good for coding. Well... except all your data going to China. I just use it for personal projects.

Yup, last month I did ~150mil tokens on DeepSeek v4 Pro for just under $3
Post reply on HN