Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

351–360 of 909 posts

Re: Open models by OpenAI

#351

Post by Matsta »

genpfault wrote: post #238
mythz wrote: post #168

Getting great performance running gpt-oss on 3x A4000's: gpt-oss:20b = ~46 tok/s More than 2x faster than my previous leading OSS models: mistral-small3.2:24b = ~22 tok/s gemma3:27b = ~19.5 tok/s Strangely getting nearly the opposite performance running on 1x 5070 Ti: mistral-small3.2:24b = ~39 tok/s gpt-oss:20b = ~21 tok/s Where gpt-oss is nearly 2x slow vs mistral-small 3.2.

Seeing ~70 tok/s on a 7900 XTX using Ollama.

I'm getting around 90 tok/s on a 3090 using Ollama.

Pretty impressive

Re: Open models by OpenAI

#352

Post by sdenton4 »

dns_snek wrote: post #214
sdenton4 wrote: post #49

Earlier quoted context omitted.

"If it is still possible to put a goalpost somewhere - and we don't care where - then it's not AGI."

LLMs are what they are, calling them "AGI" won't make them any more useful or exciting than they are, it's just going to devalue the term "AGI" which has revolutionary, disease-curing, humanity-saving connotations. What are you looking for us to say exactly? 1. We aren't even close to AGI and it's unclear that we'll ever get there, but it would change the course of humanity in a significant way if we ever do. 2. Wow…

I think getting out of the binary is good for the long run. We have something which is artificial, intelligent, and general in scope. We're there. Is it perfect? No. Is it even good? Sometimes! Do airplanes flap their wings? Also no, but they do a lot of stuff nonetheless.

Re: Open models by OpenAI

#353

Post by asadm »

highfrequency wrote: post #349
foundry27 wrote: post #154

Model cards, for the people interested in the guts: https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7... In my mind, I’m comparing the model architecture they describe to what the leading open-weights models (Deepseek, Qwen, GLM, Kimi) have been doing. Honestly, it just seems “ok” at a technical level: - both models use standard Grouped-Query Attention (64 query heads, 8 KV heads). The card talks about how…

I would guess the “secret sauce” here is distillation: pretraining on an extremely high quality synthetic dataset from the prompted output of their state of the art models like o3 rather than generic internet text. A number of research results have shown that highly curated technical problem solving data is unreasonably effective at boosting smaller models’ performance. This would be much more efficient than relying…

> research results have shown that highly curated technical problem solving data is unreasonably effective at boosting smaller models’ performance.

same seems to be true for humans

Re: Open models by OpenAI

#354

Post by lukax »

mtlynch wrote: post #347
numpad0 wrote: post #337

Here's a pair of quick sanity check questions I've been asking LLMs: "家系ラーメンについて教えて", "カレーの作り方教えて". It's a silly test but surprisingly many fails at it - and Chinese models are especially bad with it. The commonalities between models doing okay-ish for these questions seem to be Google-made OR >70b OR straight up commercial(so >200B or whatever). I'd say gpt-oss-20b is in between Qwen3 30B-A3B-2507 and Gemma 3n E4b(w…

For anyone else curious, the Chinese translates to: > "Tell me about Iekei Ramen", "Tell me how to make curry".

Japanese, not Chinese

Re: Open models by OpenAI

#355

Post by strangecasts »

SV_BubbleTime wrote: post #329
strangecasts wrote: post #139

Earlier quoted context omitted.

A small part of me is considering going from a 4070 to a 16GB 5060 Ti just to avoid having to futz with offloading I'd go for an ..80 card but I can't find any that fit in a mini-ITX case :(

I wouldn’t stop at 16GB right now. 24 is the lowest I would go. Buy a used 3090. Picked one up for $700 a few months back, but I think they were on the rise then. The 3000 series can’t do FP8fast, but meh. It’s the OOM that’s tough, not the speed so much.

Are there any 24GB cards/3090s which fit in ~300mm without an angle grinder?

Re: Open models by OpenAI

#357

Post by cco »

The lede is being missed imo.

gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year.

I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two.

But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding error) on my laptop. No $200/month subscription, no lakes being drained, etc.

I'm blown away.

Re: Open models by OpenAI

#358

Post by magoghm »

mtlynch wrote: post #347
numpad0 wrote: post #337

Here's a pair of quick sanity check questions I've been asking LLMs: "家系ラーメンについて教えて", "カレーの作り方教えて". It's a silly test but surprisingly many fails at it - and Chinese models are especially bad with it. The commonalities between models doing okay-ish for these questions seem to be Google-made OR >70b OR straight up commercial(so >200B or whatever). I'd say gpt-oss-20b is in between Qwen3 30B-A3B-2507 and Gemma 3n E4b(w…

For anyone else curious, the Chinese translates to: > "Tell me about Iekei Ramen", "Tell me how to make curry".

It's not Chinese, it's Japanese.

Re: Open models by OpenAI

#360

Post by BoorishBears »

MutedEstate45 wrote: post #266
BoorishBears wrote: post #190

Earlier quoted context omitted.

OpenAI and Azure both have zero retention options, and the NYT saga has given pretty strong confirmation they meant it when they said zero.

I think you're conflating real-time monitoring with data retention. Zero retention means OpenAI doesn't store user data, but they can absolutely still filter content, rate limit and block harmful prompts in real-time without retaining anything. That's processing requests as they come in, not storing them. The NYT case was about data storage for training/analysis not about real-time safety measures.

Ok you're off in the land of "what if" and I can just flat out say: If you have a ZDR account there is no filtering on inference, no real-time moderation, no blocking.

If you use their training infrastructure there's moderation on training examples, but SFT on non-harmful tasks still leads to a complete breakdown of guardrails very quickly.

Post reply on HN