601–610 of 909 posts
If you’re a company building AI into your product right now I think you would be irresponsible to not investigate how much you can do on open weights models. The big AI labs are going to pull the ladder up eventually, building your business on the APIs long term is foolish. These open models will always be there for you to run though (if you can get GPUs anyway).
Earlier quoted context omitted.
I tried 20b locally and it couldn't reason a way out of a basic river crossing puzzle with labels changed. That is not anywhere near SOTA. In fact it's worse than many local models that can do it, including e.g. QwQ-32b.
Well river crossings are one type of problem. My real world problem is proofing and minor editing of text. A version installed on my portable would be great.
See: https://github.com/google-ai-edge/gallery/releases/tag/1.0.3
The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…
I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…
A small LLM can do RAG, call functions, summarize, create structured data from messy text, etc... You know, all the things you'd do if you were making an actual app with an LLM.
Yeah, chat apps are pretty cheap and convenient for users who want to search the internet and write text or code. But APIs quickly get expensive when inputting a significant amount of tokens.
Earlier quoted context omitted.
Oh absolutely, AI labs certainly talk their books, including any safety angles. The controversy/outrage extended far beyond those incentivized companies too. Many people had good faith worries about Llama. Open-weight models are now vastly more powerful than Llama-1, yet the sky hasn't fallen. It's just fascinating to me how apocalyptic people are. I just feel lucky to be around in what's likely the most important de…
About 7% of people who have ever lived are alive today. Still pretty lucky, but not quite winning the lottery.
Earlier quoted context omitted.
Others have already said it, but it needs to be said again: Good god, stop treating LLMs like oracles. LLMs are not encyclopedias. Give an LLM the context you want to explore, and it will do a fantastic job of telling you all about it. Give an LLM access to web search, and it will find things for you and tell you what you want to know. Ask it "what's happening in my town this week?", and it will answer that with the…
It's fine to expect it to not know things, but the complaint is that it makes zero indication that it's just making up nonsense, which is the biggest issue with LLMs. They do the same thing when creating code.
Earlier quoted context omitted.
> It's apache2.0, so by definition it's open source. That's not true by any of the open source definitions in common use. Source code (and, optionally, derived binaries) under the Apache 2.0 license are open source. But compiled binaries (without access to source) under the Apache 2.0 license are not open source, even though the license does give you some rights over what you can do with the binaries. Normally the qu…
Please see the detailed response to a sibling post. tl;dr; weights are not binaries.
What matters (for all common definitions of open source): Are the files in "source form" (which has a definition), or are they "derived works" of the source form?
Going back to Apache 2.0. Although that doesn't define "open source", it provides legal definitions of source and non-source, which are similar to the definitions used in other open source licenses.
As you can see below, for Apache 2.0 it doesn't matter whether something is a "binary", "weights" or something else. What matters is whether it's the "preferred form for making modifications" or a "form resulting from mechanical transformation or translation". My highlights are capitalized:
- Apache License Version 2.0, January 2004
- 1. Definitions:
- "Source" form shall mean the PREFERRED FORM FOR MAKING MODIFICATIONS, including BUT NOT LIMITED TO software source code, documentation source, and configuration files.
- "Object" form shall mean any form resulting from MECHANICAL TRANSFORMATION OR TRANSLATION of a Source form, including BUT NOT LIMITED TO compiled object code, generated documentation, and conversions to other media types.
Earlier quoted context omitted.
>Google started shipping 16GB as the _lowest_ amount of RAM you can get on your Pixel 9. The Pixel 9 has 12GB of RAM[0]. You probably meant the Pixel 9 Pro. [0]: https://www.gsmarena.com/google_pixel_9-13219.php
Still an absurd amount of RAM for a phone, imo
The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…
I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…
It's a pretty bad deal. So it must be that all other things aren't equal, and I suppose the big one is hardware. But neural net based systems always have a point of sharply diminishing returns, which we seem to have unambiguously hit with LLMs already, while the price of hardware is constantly decreasing and its quality increasing. So as we go further into the future, the practicality of running locally will only increase.
Earlier quoted context omitted.
I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…
I can provide a real-world example: Low-latency code completion. The JetBrains suite includes a few LLM models on the order of a hundred megabytes. These models are able to provide "obvious" line completion, like filling in variable names, as well as some basic predictions, like realising that the `if let` statement I'm typing out is going to look something like `if let Some(response) = client_i_just_created.foobar()…
Just posted my initial impressions, took a couple of hours to write them up because there's a lot in this release! https://simonwillison.net/2025/Aug/5/gpt-oss/ TLDR: I think OpenAI may have taken the medal for best available open weight model back from the Chinese AI labs. Will be interesting to see if independent benchmarks resolve in that direction as well. The 20B model runs on my Mac laptop using less than 15GB…
That's just straight up not the case. Not sure how you can jump to that conclusion not least when you stated that you haven't tested tool calling in your post too.
Many people in the community are finding it substantially lobotomized to the point that there are "safe" memes everywhere now. Maybe you need to develop better tests that and pay more attention to benchmaxxing.
There are good things that came out of these release from OpenAI but we'd appreciate more objective analyses...