291–300 of 424 posts
Earlier quoted context omitted.
Yup, last month I did ~150mil tokens on DeepSeek v4 Pro for just under $3
Out of interest, are you using the DeepSeek plan? (I've been using it via OpenRouter and it's much more than that, but still cheap).
Earlier quoted context omitted.
Explain.
I'm not who you responded to and I don't have any info on Google. Nor can I explain in detail due to NDAs. But multiple major players are working on something along the lines of what the parent is alluding to. The "edge" AI landscape (in particular, what you can do with ~5W) is going to be nuts in about 18 months.
It says Luna is fastest, but doesn't it take way more steps to get the same job done? https://deepswe.datacurve.ai/ - (See the Agent Steps view) Or is the output speed so much higher that it cancels out? I don't see a lot of benchmarks that record actual time. But on AA, Sol on Low beats Luna on High for Time Per Task.
"Half the money I spend on advertising is wasted; the trouble is I don't know which half." -John Wanamaker This applies even more strongly to model choosing. I know for a fact that majority of my work doesn't require a very strong model, but separating the trivial and non-trivial tasks is a famously hard problem (if at all decidable).
I get frustrated with a poor quality model leaving my codebase littered with wrong comments, which then later trip up smarter models.
Earlier quoted context omitted.
If you have agents and users, you can run evals and see how far the models go. Luna is not greatest in tool calls, but if you define your problem well and the tools well, it is comparable to Gemini 4 Flash with much lower price tag.
Luna is good as an end user model for simple tasks like classification, but not as a coding model. Also do you mean Gemini 3.6 Flash? 4 doesn't exist, and Gemma 4 exists but doesn't have a Flash option.
"Half the money I spend on advertising is wasted; the trouble is I don't know which half." -John Wanamaker This applies even more strongly to model choosing. I know for a fact that majority of my work doesn't require a very strong model, but separating the trivial and non-trivial tasks is a famously hard problem (if at all decidable).
Luna is comparable to GPT 5.4 from 4 months ago on many benchmarks. I know many who have said during that time, myself included, that if that's the model they had to use for the rest of their lives, they'd be fine. GPT 5.4 is/was a very capable model.
> Starting today, GPT‑5.6 Luna, our fastest and most affordable model, will cost 80% less, I don't have the words. I genuinely thought we were in a stage where we were plateauing and going in for 5-10% improvements over months. Seeing spikes like this makes me question about where the floor really is.
I think there's also a new generation of hardware in the past year or so tuned specifically for LLM workloads, where it was almost an accident that GPUs worked to run LLMs before. So, while there's still this ridiculous shortage of hardware, what is being delivered is much faster and cheaper to run for these specific workloads.
I wasn't expecting it to happen from the US vendors, though, as they've spent so much capital to get to where they are they need to make huge margins on inference to pay it all back. I expected the Chinese models who're running much leaner operations to be the "frontier" on costs (and they have been). But, I'm glad to see OpenAI joining the "cheap and cheerful" models party. There's a lot of work in that area of capability. Probably most work people are doing falls into that area of capability.
Earlier quoted context omitted.
It's crazy. Are they doing any precomputing as you type, I wonder if you paste a block of text is it the same speed.
No, it really does take ~0.03s to generate the answer. Try your browser's developer tools and watch the requests.
I find myself getting caught up in the sheer speed of modern computing and networking. The fact I can play an online game with 10 other people is just insane.
Earlier quoted context omitted.
Experienced similar between 5.4-mini vs 5.6-luna in our own pipelines but after spending some time on prompt optimization and testing out various reasoning effort levels 5.6-luna was well worth it. Did you just replace model selection while keeping everything else in place or spend some time on evaling with newer prompts etc?
No we kept prompts as is, just swapped model. The prompt is already quite optimized for the task. How would updating it possibly make a more intelligent model spend less tokens than a less intelligent model? Care to elaborate?
Earlier quoted context omitted.
No, it really does take ~0.03s to generate the answer. Try your browser's developer tools and watch the requests.
Isn't it crazy that we can send a message, across the world near instantaneously and have a coherent reply, generated by a computer, sent back to your screen, in under 500ms in most circumstances. I find myself getting caught up in the sheer speed of modern computing and networking. The fact I can play an online game with 10 other people is just insane.