Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

191–200 of 909 posts

Re: Open models by OpenAI

#191

Post by henriquegodoy »

Seeing a 20B model competing with o3's performance is mind blowing like just a year ago, most of us would've called this impossible - not just the intelligence leap, but getting this level of capability in such a compact size.

I think that the point that makes me more excited is that we can train trillion-parameter giants and distill them down to just billions without losing the magic. Imagine coding with Claude 4 Opus-level intelligence packed into a 10B model running locally at 2000 tokens/sec - like instant AI collaboration. That would fundamentally change how we develop software.

Re: Open models by OpenAI

#192

Post by liuliu »

rfoo wrote: post #176
foundry27 wrote: post #154

Model cards, for the people interested in the guts: https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7... In my mind, I’m comparing the model architecture they describe to what the leading open-weights models (Deepseek, Qwen, GLM, Kimi) have been doing. Honestly, it just seems “ok” at a technical level: - both models use standard Grouped-Query Attention (64 query heads, 8 KV heads). The card talks about how…

Or, you can say, OpenAI has some real technical advancements on stuff besides attn architecture. GQA8, alternating SWA 128 / full attn do all seem conventional. Basically they are showing us that "no secret sauce in model arch you guys just sucks at mid/post-training", or they want us to believe this. The model is pretty sparse tho, 32:1.

Kimi K2 paper said that the model sparsity scales up with parameters pretty well (MoE sparsity scaling law, as they call, basically calling Llama 4 MoE "done wrong"). Hence K2 has 128:1 sparsity.

Re: Open models by OpenAI

#194

Post by Nimitz14 »

shpongled wrote: post #143
m_ke wrote: post #104

Earlier quoted context omitted.

No they’re usually done at each attention layer.

Do you know when this was introduced (or which paper)? AFAIK it's not that way in the original transformer paper, or BERT/GPT-2

This is normal. Rope was introduced after bert/gpt2

Re: Open models by OpenAI

#195

Post by hnuser123456 »

minimaxir wrote: post #20

I'm disappointed that the smallest model size is 21B parameters, which strongly restricts how it can be run on personal hardware. Most competitors have released a 3B/7B model for that purpose. For self-hosting, it's smart that they targeted a 16GB VRAM config for it since that's the size of the most cost-effective server GPUs, but I suspect "native MXFP4 quantization" has quality caveats.

Native FP4 quantization means it requires half as many bytes as parameters, and will have next to zero quality loss (on the order of 0.1%) compared to using twice the VRAM and exponentially more expensive hardware. FP3 and below gets messier.

Re: Open models by OpenAI

#198

Post by nirav72 »

I don't exactly have the ideal hardware to run locally - but just ran the 20b in LMStudio with a 3080 Ti (12gb vram) with some offloading to CPU. Ran couple of quick code generation tests. On average about 20t/sec. But response quality was very similar or on-par with chatgpt o3 for the same code it outputted. So its not bad.

Re: Open models by OpenAI

#199

Post by nodesocket »

Anybody got this working in Ollama? I'm running latest version 0.11.0 with WebUI v0.6.18 but getting:

> List the US presidents in order starting with George Washington and their time in office and year taken office.

>> 00: template: :3: function "currentDate" not defined

Post reply on HN