GPT-5.6 Sol. 5.7 Luna. 5.8 Mars. Meanwhile my code still runs on GPT-3.5 and nobody noticed.
Previewing GPT‑5.6 Sol: a next-generation model
601–610 of 797 posts
Re: Previewing GPT‑5.6 Sol: a next-generation model
#602Earlier quoted context omitted.
For comparison, openrouter says opus 4.8 is ~55 tokens/s and fast mode is ~102. 750 tokens/s for their largest model is going to be nuts
What about 15k tokens per second? [0] I remember looking at this earlier in the year and it being so fast that it feels fake. And, yes, this model is old - but still awesome for what it is. [0] https://chatjimmy.ai/
I just tried it, and the answer is non-sense.
I asked it something simple, list some good indie puzzle games, and half the answers are games that don't exist. Imo quality > speed.
Re: Previewing GPT‑5.6 Sol: a next-generation model
#603Here is a trend I'm noticing: - GPT-5 mini costs $0.25/$2 and will be discontinued in December. - GPT-5.4 mini costs $0.75/$4.5 and is supposed to be the replacement. - GPT-5.4 nano costs $0.2/$1.25 and, while it ranks better in benchmarks than GPT-5 mini, it's not even close when you test it in real scenarios. So you're left being forced to go to GPT 5.4 mini if you use 5 mini today. The same thing is happening here…
On Nano "it's not even close when you test it in real scenarios" - what have you seen? What kind of things can GPT-5 Mini handle that GPT-5.4 Nano cannot?
Also can confirm gpt-5.4-nano was unable to even keep up with 4.1-mini. Had to move off of OpenAI once 4.1-mini was retired
Re: Previewing GPT‑5.6 Sol: a next-generation model
#604Easily the most interesting part of this announcement is buried in the second to last paragraph: "We're also launching GPT‑5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed. Access will initially be limited to select customers as we expand capacity." 750 tokens/s on a frontier model is going to be extremely interesting. I doubt this new vers…
This is a strange one. We know the hardware capabilities of Cerebras force them to do aggressive REAP pruning to serve Kimi K2.6. Meaning that about 750B parameters is the upper limit of what they can serve economically. Not sure if this means Sol is smaller than anyone thinks or that they're just going to charge so much that a very inefficient serving regime is feasible.
Re: Previewing GPT‑5.6 Sol: a next-generation model
#605All of these LLMs are getting better at being at an LLM But GPT-5.5 is as useful an LLM can be; it has solved lemmas I've thought about for a year, it can implement typed STLCs in Rust when I give it a formal grammar, it can help me analyze Postgres planner dumps. It's great at tasks that have short solutions but - they cannot learn based on a project - their long term planning capabilities are worse than worms - the…
Fable had very confident decision-making and would push past obstacles that Opus finds daunting :(
Re: Previewing GPT‑5.6 Sol: a next-generation model
#606GPT-5.6 Sol. 5.7 Luna. 5.8 Mars. Meanwhile my code still runs on GPT-3.5 and nobody noticed.
Weird flex. There is cheaper better and faster models you could of moved to with an hour effort
Re: Previewing GPT‑5.6 Sol: a next-generation model
#607GPT-5.6 Sol. 5.7 Luna. 5.8 Mars. Meanwhile my code still runs on GPT-3.5 and nobody noticed.
you can even use gpt 1.0 but it proves nothing
Re: Previewing GPT‑5.6 Sol: a next-generation model
#608Earlier quoted context omitted.
Funnily enough, pasting your comment straight into Jimmy leads to a... Funnily suboptimal answer that does not answer the question. As someone else already contributed, this is driven by a Canadian startup taalas that basically makes chips that are llms, so everything is very fast but also, baked into the chip. Once this kind of stuff is a commodity in like 10 years, our world will be very, very different.
Taalas HC1 AI uses Llama 3.1 8B, but takes up a massive 53B transistors and 815mm2 on TSMC N6 (nearly at the reticle limit of 858mm2). N2 is a little less than 3x as dense (110MTr/mm2 vs 313MTr/mm2). This chip would still be 272mm2 on N2 which is an eye-watering $30k/wafer and bigger than a 9950x or Nvidia 5070. This just isn't feasible. Some of the latest-gen LLMs seem to have 5-10T parameters or about 1000x more. I…
> Things like continuing education so your model knows about the latest NPM packages or world news is super important, but seems like it would require new chips.
They probably have a few ideas around that. Me, personally, I'd have one main expensive chip (replaced every 10 years, or whatever), with a secondary cheap chip in front of it that gets replaced every year or so.
The secondary chip could act the way RAG does, or perhaps both chips together can act as LoRA.
Either way, 99.999% of the knowledge is static, you just need to fine-tune the weights with that remaining 0.001% knowledge, which can be done using RAG or LoRA on a much smaller (thus cheaper) disposable chip.
Re: Previewing GPT‑5.6 Sol: a next-generation model
#609Here is a trend I'm noticing: - GPT-5 mini costs $0.25/$2 and will be discontinued in December. - GPT-5.4 mini costs $0.75/$4.5 and is supposed to be the replacement. - GPT-5.4 nano costs $0.2/$1.25 and, while it ranks better in benchmarks than GPT-5 mini, it's not even close when you test it in real scenarios. So you're left being forced to go to GPT 5.4 mini if you use 5 mini today. The same thing is happening here…
> Maybe it’s the realization that it was never that cheap in the first place and they're forcing us to upgrade in a slow and painful way. All the analysis I have seen points to frontier models being profitable to serve. It’s using 50% or more of your GPUs for research plus CapEx for capacity expansion that makes these businesses so heavily cash-negative. What you are observing is downstream of another detail. It gets…
There is really ample analysis pointing to inference not being profitable, look at anything Ed Zitron has reported.
Re: Previewing GPT‑5.6 Sol: a next-generation model
#610Terra and Luna? Last time I had heard that, it didn’t end quite well