771–780 of 909 posts
Careful, this model tries to connect to the Internet. No idea what it's doing. https://crib.social/notice/AwsYxAOsg1pqAPLiHA
The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…
I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…
For example, "generate a heatmap of each token/word and how 'unexpected' they are" or "find me a prompt that creates the closest match to this text"
To be efficient both require access that is not exposed over API.
Just posted my initial impressions, took a couple of hours to write them up because there's a lot in this release! https://simonwillison.net/2025/Aug/5/gpt-oss/ TLDR: I think OpenAI may have taken the medal for best available open weight model back from the Chinese AI labs. Will be interesting to see if independent benchmarks resolve in that direction as well. The 20B model runs on my Mac laptop using less than 15GB…
Nice write up! One test I do is to give a common riddle but word it slightly to see if it can actually reason. For example: "Bobs dad has five daughters, Lala, Lele, Lili, Lolo and ???" The 20B model kept picking the answer of the original riddle, even after explaining extra information to it. The original riddle is: "Janes dad has five daughters, Lala, Lele, Lili, Lolo and ???"
Earlier quoted context omitted.
I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…
People like myself that firmly believe there will come a time, possibly very soon that all these companies (OpenAI, Anthropic etc) will raise their prices substantially. By then no one will be able to do their work to the standard expected of them without AI, and by then maybe they charge $1k per month, maybe they charge $10k. If there is no viable alternative the sky is the limit. Why do you think they continue to r…
I'm curious to hear what you're trying to run, because I haven't used any software that is not compatible with multiple GPUs.
Earlier quoted context omitted.
> In fact it's worse than many local models that can do it, including e.g. QwQ-32b. I'm not going to be surprised that a 20B 4/32 MoE model (3.6B parameters activated) is less capable at a particular problem category than a 32B dense model, and its quite possible for both to be SOTA, as state of the art at different scale (both parameter count and speed which scales with active resource needs) is going to have differ…
[flagged]
Earlier quoted context omitted.
You’re in a bubble. It was no surprise to folks who touch grass on the regular.
> You’re in a bubble. Sure, all I have to go on from the other side of the Atlantic is the internet. So in that regard, kinda like the AI. One of the big surprises from the POV of me in Jan 2024, is that I would have anticipated Trump being in prison and not even available as an option for the Republican party to select as a candidate for office, and that even if he had not gone to jail that the Republicans would not…
It's the White House that wanted Trump to be candidate. They played Republican primary voters like a fiddle by launching a barrage of transparently political prosecutions just as Republican primaries were starting.
And then they still lost the general election.
Earlier quoted context omitted.
Privacy, both personal and for corporate data protection is a major reason. Unlimited usage, allowing offline use, supporting open source, not worrying about a good model being taken down/discontinued or changed, and the freedom to use uncensored models or model fine tunes are other benefits (though this OpenAI model is super-censored - “safe”). I don’t have much experience with local vision models, but for text ques…
+1 - I work in finance, and there's no way we're sending our data and code outside the organization. We have our own H100s.
Earlier quoted context omitted.
My main goal with that benchmark is to see if it can produce HTML and JavaScript code that runs without errors for a moderately complex challenge. It's not a comprehensive benchmark - there are many ways you could run it in ways that would be much more informative and robust. It's great as a quick single sentence prompt to get a feeling for if the model can produce working JavaScript or not.
Not really the other commenters are correct I feel and this is not really proving anything about the fundamental capability of the model. It’s just a hello world benchmark adding no real value, just driving blog traffic for you.
Comparing them between models is also kind of interesting, even if it's not a flawlessly robust comparison: https://simonwillison.net/tags/space-invaders/