761–770 of 909 posts
After a good part of a year using Chinese models (which are fantastic, happy to have them) it is cool to now be relying on US models with the newest 4B Google Gemma model and now also the 20B OpenAI model for running locally.
Earlier quoted context omitted.
I tried 20b locally and it couldn't reason a way out of a basic river crossing puzzle with labels changed. That is not anywhere near SOTA. In fact it's worse than many local models that can do it, including e.g. QwQ-32b.
Well river crossings are one type of problem. My real world problem is proofing and minor editing of text. A version installed on my portable would be great.
Just posted my initial impressions, took a couple of hours to write them up because there's a lot in this release! https://simonwillison.net/2025/Aug/5/gpt-oss/ TLDR: I think OpenAI may have taken the medal for best available open weight model back from the Chinese AI labs. Will be interesting to see if independent benchmarks resolve in that direction as well. The 20B model runs on my Mac laptop using less than 15GB…
Nice write up! One test I do is to give a common riddle but word it slightly to see if it can actually reason. For example: "Bobs dad has five daughters, Lala, Lele, Lili, Lolo and ???" The 20B model kept picking the answer of the original riddle, even after explaining extra information to it. The original riddle is: "Janes dad has five daughters, Lala, Lele, Lili, Lolo and ???"
Earlier quoted context omitted.
I tried 20b locally and it couldn't reason a way out of a basic river crossing puzzle with labels changed. That is not anywhere near SOTA. In fact it's worse than many local models that can do it, including e.g. QwQ-32b.
I tried the two US presidents having the same parents one, and while it understood the intent, it got caught up in being adamant that Joe Biden won the election in 2024 and anything I do to try and tell it otherwise is dismissed as being false and expresses quite definitely that I need to do proper research with legitimate sources.
I use the SOTA models from Google and OpenAI mostly for getting feedback on ideas, helping me think through designs, and sometimes for coding.
Your question is clearly best answered using a large commercial model with a web search tool. That said, integrating a local model with a home built interface to something like the Brave search API can be effective but I no longer make the effort.
Earlier quoted context omitted.
https://skinflint.co.uk/?cat=gra16_512&hloc=uk&v=e&hloc=at&h... 5070 Ti Super will also have 24GB.
Oh nice, thank you :) Admittedly a little tempting to see how the 5070 Ti Super shakes out!
50xx series supports MXFP4 format, but I'm not sure about 3090.
Earlier quoted context omitted.
I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…
Privacy, both personal and for corporate data protection is a major reason. Unlimited usage, allowing offline use, supporting open source, not worrying about a good model being taken down/discontinued or changed, and the freedom to use uncensored models or model fine tunes are other benefits (though this OpenAI model is super-censored - “safe”). I don’t have much experience with local vision models, but for text ques…
Someone else responded to you about working for a financial organization and not using public APIs - another great use case.
I ended up blown away. via Cerebras/Groq, you're looking at around 1000 tok/sec for the 120B model. For gentic code generation, I found the abilities to exceed gpt-4.1. Tool calling was surprisingly good, albeit not as good as Qwen3 Coder for me.
It's a very capable model, and a very good release. The high throughput is a game changer.
* It seems that using version @lmstudio's 20B gguf version (https://huggingface.co/lmstudio-community/gpt-oss-20b-GGUF) will have options for reasoning effort.
* My MBP M1 16GB config: temp 0.8, max content length 7990, GPU offload 8/24, runs slow and still fine for me.
* I tried testing with MCP with the above config, with basic tools like time and fetch + reasoning effort low, and the tool calls instruction follow is quite good.
* In LM Studio's Developer tab there is a log output about the model information which is useful to learn.
Overall, I like the way OpenAI backs to being Open AI, again, after all those years.
--
Shameless plug, If anyone want to try out gpt-oss-120b and gpt-oss-20b as alternative to their own demo page [0], I have added both models with OpenRouter providers in VT Chat [1] as real product. You can try with an OpenRouter API Key.
Earlier quoted context omitted.
Why do any compute locally? Everything can just be cloud based right? Won't that work much better and scale easily? We are not even at that extreme and you can already see the unequal reality that too much SaaS has engendered
Comcast comes to mind ;-)
Earlier quoted context omitted.
You’re in a bubble. It was no surprise to folks who touch grass on the regular.
> You’re in a bubble. Sure, all I have to go on from the other side of the Atlantic is the internet. So in that regard, kinda like the AI. One of the big surprises from the POV of me in Jan 2024, is that I would have anticipated Trump being in prison and not even available as an option for the Republican party to select as a candidate for office, and that even if he had not gone to jail that the Republicans would not…