371–380 of 424 posts
I would pay significantly more to use these models if there was a legal contract that guaranteed they weren't ever terfing them and some way to prove that.
Has been pretty accurate when pre-release quality or bugs pop up. In theory you could run your own.
What a time to be alive, the fall of the Chinese edge
DeepSeek may suffer from this... The official version has still not arrived, and Luna now costs only 70% of V4 Pro. Just imagine how much the same level of intelligence cost only a few months ago.
https://api-docs.deepseek.com/updates/
The update seems to say that it's a re-post train of exactly the same model.
Earlier quoted context omitted.
If you optimize program A and manage to wring out a 1% improvement, and I optimize program B and improve performance by 20%, you can see the problem with trying to infer anything from those two numbers. Edit: searching for the story now, further bolstering the point is that was 1% in training time [1], and the openAI claim is 20% in end to end inference cost . This is a bad comparison. [1] https://deepmind.google/blo…
> This is a bad comparison. How so? First, kernel writing (or ML engineering more broadly) is a highly specialised task. Not everyone can do it. It shows that models are getting better and better at (easily verifiable) hard tasks. And you can "hire" that expertise much easier than you can hire the equivalent meatbags. And more importantly you can "fire" them as soon as the task is done. And then hire them 3 months la…
Without looking at the specific changes made, it is impossible to know whether this represents a real capability improvement. For example, maybe the 20% gain was due to a very obvious/easy to catch inefficiency while the 1% improvement was due to something subtle.
> Starting today, GPT‑5.6 Luna, our fastest and most affordable model, will cost 80% less, I don't have the words. I genuinely thought we were in a stage where we were plateauing and going in for 5-10% improvements over months. Seeing spikes like this makes me question about where the floor really is.
it's 80% less cost, not 80% in efficiency gains, could be that Luna was overpriced to begin with, we don't have much info on the models themselves. Assuming the efficiency gains are real, I feel like something has to give, maybe worse quality due to aggressive quantization/kv cache compression?
Earlier quoted context omitted.
how many times do you have to be metaphorically hit in the head with a brick before you realize inference margins at api pricing were 80%+
So, does that mean they're selling tokens at cost ? Or that they realized they were doing 180% more work than necessary and they're passing the savings on because "your margin is my opportunity" and this is a dog-eat-dog fight, but 80% margins are nice, no one needs multiple hundreds?
Earlier quoted context omitted.
how many times do you have to be metaphorically hit in the head with a brick before you realize inference margins at api pricing were 80%+
Uh… are you okay? That’s a completely disproportionate, aggressive tone. How is that warranted
1. The basic mechanism is literally described in the post: they found efficiencies and passed it down
2. This has been the trend for all the time LLMs have existed
Why confusion then? What’s surprising you?
"Half the money I spend on advertising is wasted; the trouble is I don't know which half." -John Wanamaker This applies even more strongly to model choosing. I know for a fact that majority of my work doesn't require a very strong model, but separating the trivial and non-trivial tasks is a famously hard problem (if at all decidable).
I remember bugs always end up being more expensive then the first time I implemented the feature… so for me always sol always max… I don’t want to pay for a bug later
You use the big models to plan. Not just the overall plan, but which files need to be edited etc. Then they give that to the lower end model. So Luna or Sonnet, which are perfectly capable of following instructions and still creative enough to not get stuck.
Earlier quoted context omitted.
It's funny how codex itself can't do Sol orchestrator / Luna implementor out of the box.
That's surprising. Does it have no sub agent support at all or does it just use the same agent as the parent?
(also you need to gate it with "tell the subagent it's a subagent" and "if you are a subagent, don't spawn subagents" or you'll get a matroshka doll of sonnets all the way down =P )
Codex kinda sorta can launch a subagent, but that's about it.