161–170 of 179 posts
I think the end game is decent local model that does 80% of the work, and that also knows when to call the cloud, and which models to call.
For development use cases, I switched to Sonnet 4.5 and haven't looked back. I mean, sure, sometimes I also use GPT-5 (and mini) and Gemini 2.5 Pro (and Flash), and also Cerebras Code just switched to providing GLM 4.6 instead of the previous Qwen3 Coder so those as well, but in general the frontier models are pretty good for development and I wouldn't have much reason to use something like Sonnet 4 or 3.7 or whateve…
Yeah, I'm just going through the Cerebras migration at the moment. It's a shame Cerebras completely dropped Qwen3 Coder's fast tool calling, short and instant responses, and better speed overall for GLM 4.6 thinking. Qwen3 is really good at hitting the tools first, then coming up with a well-grounded answer based on reality. Sometimes it's good when a model is Socratic: just knows it knows nothing. GLM 4.6 on the oth…
I don't know, I had a lot of issues with Qwen models when it comes to RooCode/Cline - failed edits (albeit with a requirement for 100% precision, since I don't want the wrong lines to be replaced) or calling tools without parameters (e.g. list_files without path) and also stuff like using wrong path separators or using the wrong commands for the shell that's available (e.g. cmd when Git Bash is the shell).
GLM 4.6 seems better in that regard so far, maybe the coming weeks and months will show that better.
Earlier quoted context omitted.
That would be great if that was the case but my understanding is that the progress is plateauing. I don't know how much of this is anthorpic / Google / openAI holding itself back to save money and how much is the state of the art improvement slowing down though. I can imagine there could be a 64 GB GPU in five years as absurd as it feels to type that today.
> a 64 GB GPU in five years Is there a digit missing? I don't understand why this existing in 5 years is absurd
I've found that the VSCode GitHub Copilot extension defaults to Claude Sonnet 4.0 (in agent mode) in all new workspaces. It's the first thing I check now, but I imagine a lot of people just roll with it, especially if they use inline completions where it might not be obvious what model is being used.
Earlier quoted context omitted.
Most people can’t affort the GPUs for local models if you want to get close to cloud capabilities.
A 4090 has 24GB of VRAM allowing you to run a 22B model entirely in memory at FP8 and 24B models at Q6_K (~19GB). A 5090 has 32GB of VRAM allowing you to run a 32B model in memory at Q6_K. You can run larger models by splitting the GPU layers that are run in VRAM vs stored in RAM. That is slower, but still viable. This means that you can run the Qwen3-Coder-30B-A3B model locally on a 4090 or 5090. That model is a Mix…
I spent last weekend experimenting with Ollama and LM studio. I was impressed at how good Qwen3-Coder is. Not as good as Claude, but close - maybe even better in some ways.
As I understand it, the latest Macs are good for local LLMs due to their unified memory. 32GB of RAM in one of the newer M-series seems to be the "sweet spot" for price versus performance.
Earlier quoted context omitted.
That would be great if that was the case but my understanding is that the progress is plateauing. I don't know how much of this is anthorpic / Google / openAI holding itself back to save money and how much is the state of the art improvement slowing down though. I can imagine there could be a 64 GB GPU in five years as absurd as it feels to type that today.
What gives you the impression the progress is plateauing? I'm finding the difference just between Sonnet 4 and Sonnet 4.5 to be meaningful in terms of the complexity of tasks I'm willing to use them for.
That doesn't mean "not plateauing".
It's better, certainly, but the difference between SOTA now and SOTA 6 months ago is a fraction of the difference between SOTA 6 months ago and the difference 18 months ago.
It doesn't mean that the models aren't getting better, it means that the improvement in each generation is smaller than the the improvement in the previous generation.
Earlier quoted context omitted.
Today. But what about in 5 years? Would you bet we will be paying hundreds of billions to OpenAI yearly or buying consumer GPUs? I know what I will be doing.
Paying for compute in the cloud. That’s what I am betting on. Multiple providers, different data center players. There may be healthy margins for them but I would bet it’s always going to be relatively cheaper for me to pay for the compute rather than manage it myself.
Depends almost completely on usage. No one is renting out hardware 24x7 and making a loss on it.
If you only have sporadic use then renting is better. If you're running it almost all the time of purchasing it outright is better.
Earlier quoted context omitted.
What gives you the impression the progress is plateauing? I'm finding the difference just between Sonnet 4 and Sonnet 4.5 to be meaningful in terms of the complexity of tasks I'm willing to use them for.
> I'm finding the difference just between Sonnet 4 and Sonnet 4.5 to be meaningful in terms of the complexity of tasks I'm willing to use them for. That doesn't mean "not plateauing". It's better, certainly, but the difference between SOTA now and SOTA 6 months ago is a fraction of the difference between SOTA 6 months ago and the difference 18 months ago. It doesn't mean that the models aren't getting better, it mean…
Comparing a 12 month period to a 6 month period feels unfair to me though. I think we will have a much fuller picture by the end of the year - I have high expectations for the next wave of Chinese models and for Gemini 3.
Earlier quoted context omitted.
Paying for compute in the cloud. That’s what I am betting on. Multiple providers, different data center players. There may be healthy margins for them but I would bet it’s always going to be relatively cheaper for me to pay for the compute rather than manage it myself.
> There may be healthy margins for them but I would bet it’s always going to be relatively cheaper for me to pay for the compute rather than manage it myself. Depends almost completely on usage. No one is renting out hardware 24x7 and making a loss on it. If you only have sporadic use then renting is better. If you're running it almost all the time of purchasing it outright is better.
Earlier quoted context omitted.
> I'm finding the difference just between Sonnet 4 and Sonnet 4.5 to be meaningful in terms of the complexity of tasks I'm willing to use them for. That doesn't mean "not plateauing". It's better, certainly, but the difference between SOTA now and SOTA 6 months ago is a fraction of the difference between SOTA 6 months ago and the difference 18 months ago. It doesn't mean that the models aren't getting better, it mean…
18 months ago to 6 months ago was indeed a busy period - both multimodal image input and reasoning models were rare at the start of that time period and common by the end of it. Comparing a 12 month period to a 6 month period feels unfair to me though. I think we will have a much fuller picture by the end of the year - I have high expectations for the next wave of Chinese models and for Gemini 3.
Okay. Let me clarify then.
The difference between SOTA now and SOTA 6 months ago is a fraction of the difference between SOTA 6 months ago and SOTA 12 months ago.
That still "plateauing". The performance of the models, should you take the time to chart them, is clearly asymptotic and we're in the flattening out phase now.
I also observe that all the models are converging on roughly the same performance, which makes me think that we are approaching some maxima with the current approach.