Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

251–260 of 909 posts

Re: Open models by OpenAI

#251

Post by jph00 »

logicchains wrote: post #183
Imustaskforhelp wrote: post #151

Earlier quoted context omitted.

Okay I will be honest, I was so hyped up about This model but then I went to localllama and saw it that the: 120 B model is worse at coding compared to qwen 3 coder and glm45 air and even grok 3... ( https://www.reddit.com/r/LocalLLaMA/comments/1mig58x/gptoss1... )

It's only got around 5 billion active parameters; it'd be a miracle if it was competitive at coding with SOTA models that have significantly more.

On this bench it underperforms vs glm-4.5-air, which is an MoE with fewer total params but more active params.

Re: Open models by OpenAI

#253

Post by coolspot »

jcmontx wrote: post #182

I'm out of the loop for local models. For my M3 24gb ram macbook, what token throughput can I expect? Edit: I tried it out, I have no idea in terms of of tokens but it was fluid enough for me. A bit slower than using o3 in the browser but definitely tolerable. I think I will set it up in my GF's machine so she can stop paying for the full subscription (she's a non-tech professional)

40 t/s

Re: Open models by OpenAI

#254

Post by steinvakt2 »

pamelafox wrote: post #142
pamelafox wrote: post #96

Anyone tried running on a Mac M1 with 16GB RAM yet? I've never run higher than an 8GB model, but apparently this one is specifically designed to work well with 16 GB of RAM.

Update: I tried it out. It took about 8 seconds per token, and didn't seem to be using much of my GPU (MPU), but was using a lot of RAM. Not a model that I could use practically on my machine.

Did you run it the best way possible? im no expert, but I understand it can affect inference time greatly (which format/engine is used)

Re: Open models by OpenAI

#255

Post by rushingcreek »

logicchains wrote: post #178
foundry27 wrote: post #154

Model cards, for the people interested in the guts: https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7... In my mind, I’m comparing the model architecture they describe to what the leading open-weights models (Deepseek, Qwen, GLM, Kimi) have been doing. Honestly, it just seems “ok” at a technical level: - both models use standard Grouped-Query Attention (64 query heads, 8 KV heads). The card talks about how…

>They did something to quantize >90% of the model parameters to the MXFP4 format (4.25 bits/parameter) to let the 120B model to fit on a single 80GB GPU, which is pretty cool They said it was native FP4, suggesting that they actually trained it like that; it's not post-training quantisation.

The native FP4 is one of the most interesting architectural aspects here IMO, as going below FP8 is known to come with accuracy tradeoffs. I'm curious how they navigated this and how the FP8 weights (if they exist) were to perform.

Re: Open models by OpenAI

#256

Post by ramoz »

This is a solid enterprise strategy.

Frontier labs are incentivized to start breaching these distribution paths. This will evolve into large scale "intelligent infra" plays.

Re: Open models by OpenAI

#257

Post by sipjca »

davidw wrote: post #241

Big picture, what's the balance going to look like, going forward between what normal people can run on a fancy computer at home vs heavy duty systems hosted in big data centers that are the exclusive domain of Big Companies? This is something about AI that worries me, a 'child' of the open source coming of age era in the 90ies. I don't want to be forced to rely on those big companies to do my job in an efficient way…

Isn’t it that hardware catches up and becomes cheaper? The margin on these chips right now is outrageous, but what happens as there is more competition? What happens when there is more supply? Are we overbuilding? Apple M series chips already perform phenomenally for this class of models and you bet both AMD and NVIDIA are playing with unified memory architectures too for the memory bandwidth. It seems like today’s really expensive stuff may become the norm rather than the exception. Assuming architectures lately stay similar and require large amounts of fast memory.

Re: Open models by OpenAI

#258

Post by turnsout »

pamelafox wrote: post #142
pamelafox wrote: post #96

Anyone tried running on a Mac M1 with 16GB RAM yet? I've never run higher than an 8GB model, but apparently this one is specifically designed to work well with 16 GB of RAM.

Update: I tried it out. It took about 8 seconds per token, and didn't seem to be using much of my GPU (MPU), but was using a lot of RAM. Not a model that I could use practically on my machine.

To clarify, this was the 20B model?

Re: Open models by OpenAI

#259

Post by bookofjoe »

v5v3 wrote: post #232
davio wrote: post #226

Earlier quoted context omitted.

On a M1 MacBook Air with 8GB, I got this running Gemma 3n: 12.63 tok/sec • 860 tokens • 1.52s to first token I'm amazed it works at all with such limited RAM

I have started a crowdfunding to get you a MacBook air with 16gb. You poor thing.

Up the ante with an M4 chip

Re: Open models by OpenAI

#260

Post by pamelafox »

steinvakt2 wrote: post #254
pamelafox wrote: post #142

Earlier quoted context omitted.

Update: I tried it out. It took about 8 seconds per token, and didn't seem to be using much of my GPU (MPU), but was using a lot of RAM. Not a model that I could use practically on my machine.

Did you run it the best way possible? im no expert, but I understand it can affect inference time greatly (which format/engine is used)

I ran it via Ollama, which I assume uses the best way. Screenshot in my post here: https://bsky.app/profile/pamelafox.bsky.social/post/3lvobol3...

I'm still wondering why my MPU usage was so low.. maybe Ollama isn't optimized for running it yet?

Post reply on HN