31–40 of 179 posts
30 seconds-1 minute is just the time I am patient enough to wait as that's the time I am spending on writing a question.
Faster models just make too many mistakes / don't understand the question.
This data is basically meaningless, show us the latest stats.
Seems to completely ignore usage of local/free models as well as anything but Sonnet/ChatGPT. So my confidence in the good faith of the author is... heavily restricted.
Most people can’t affort the GPUs for local models if you want to get close to cloud capabilities.
There was even a recent release of Granite4 that runs on a Raspberry Pi.
https://github.com/Jewelzufo/granitepi-4-nano
For my local work I use Ollama. (M4 Max 128GB)
- gpt-oss. 20b or 120b depending on complexity of use cases.
- granite4 for speed and lower complexity (around the same as gpt20b).
Earlier quoted context omitted.
Most people can’t affort the GPUs for local models if you want to get close to cloud capabilities.
A 4090 has 24GB of VRAM allowing you to run a 22B model entirely in memory at FP8 and 24B models at Q6_K (~19GB). A 5090 has 32GB of VRAM allowing you to run a 32B model in memory at Q6_K. You can run larger models by splitting the GPU layers that are run in VRAM vs stored in RAM. That is slower, but still viable. This means that you can run the Qwen3-Coder-30B-A3B model locally on a 4090 or 5090. That model is a Mix…
GPT5 is HELLISHLY slow. That's all there is to it. It loves doing a whole bunch of reasoning steps and prolaim how mucf of a very good job it did clearing up its own todo steps and all that mumbo jumbo, but at the end of the day, I only asked it a small piece of information about nginx try_files that even GPT3 could answer instantly. Maybe before you make reasoning models that go on funny little sidequests wher they…
If you are talking about local models, you can switch that off. The reasoning is a common technique now to improve the accuracy of the output where the question is more complex.
Seems to completely ignore usage of local/free models as well as anything but Sonnet/ChatGPT. So my confidence in the good faith of the author is... heavily restricted.
Inference for new releases is routinely bugged for at least a month or two as well, depending on how active the devs of a specific inference engine are and how much model creators collaborate. Personally, I hate how data from GPT's few week (and arguably somewhat ongoing) sycophancy rampage has leaked into datasets that are used for training local models, making a lot of new LLM releases insufferable to use.
Earlier quoted context omitted.
Most people can’t affort the GPUs for local models if you want to get close to cloud capabilities.
A 4090 has 24GB of VRAM allowing you to run a 22B model entirely in memory at FP8 and 24B models at Q6_K (~19GB). A 5090 has 32GB of VRAM allowing you to run a 32B model in memory at Q6_K. You can run larger models by splitting the GPU layers that are run in VRAM vs stored in RAM. That is slower, but still viable. This means that you can run the Qwen3-Coder-30B-A3B model locally on a 4090 or 5090. That model is a Mix…
It could be true that newer models just produce more tokens seemingly out of no reasons. But with the increasing number of tool definitions, in the long run, I think it will pay off.
Just a few days ago, I read "Interleaved Thinking Unlocks Reliable MiniMax-M2 Agentic Capability"[1]. I think they have a valid point that this thinking process has significance as we are moving towards agents.
[1] https://www.minimax.io/news/why-is-interleaved-thinking-impo...