Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

221–230 of 909 posts

Re: Open models by OpenAI

#221

Post by Adrig »

ClassAndBurn wrote: post #158

Open models are going to win long-term. Anthropics' own research has to use OSS models [0]. China is demonstrating how quickly companies can iterate on open models, allowing smaller teams access and augmentation to the abilities of a model without paying the training cost. My personal prediction is that the US foundational model makers will OSS something close to N-1 for the next 1-3 iterations. The CAPEX for the fou…

I'm a layman but it seemed to me that the industry is going towards robust foundational models on which we plug tools, databases, and processes to expand their capabilities.

In this setup OSS models could be more than enough and capture the market but I don't see where the value would be to a multitude of specialized models we have to train.

Re: Open models by OpenAI

#222

Post by spott »

shpongled wrote: post #143
m_ke wrote: post #104

Earlier quoted context omitted.

No they’re usually done at each attention layer.

Do you know when this was introduced (or which paper)? AFAIK it's not that way in the original transformer paper, or BERT/GPT-2

All the Llamas have done it (well, 2 and 3, and I believe 1, I don't know about 4). I think they have a citation for it, though it might just be the RoPE paper (https://arxiv.org/abs/2104.09864).

I'm not actually aware of any model that doesn't do positional embeddings on a per-layer basis (excepting BERT and the original transformer paper, and I haven't read the GPT2 paper in a while, so I'm not sure about that one either).

Re: Open models by OpenAI

#223

Post by fnands »

Mhh, I wonder if these are distilled from GPT4-Turbo.

I asked it some questions and it seems to think it is based on GPT4-Turbo:

> Thus we need to answer "I (ChatGPT) am based on GPT-4 Turbo; number of parameters not disclosed; GPT-4's number of parameters is also not publicly disclosed, but speculation suggests maybe around 1 trillion? Actually GPT-4 is likely larger than 175B; maybe 500B. In any case, we can note it's unknown.

As well as:

> GPT‑4 Turbo (the model you’re talking to)

Re: Open models by OpenAI

#224

Post by vl »

PeterStuer wrote: post #181
artembugara wrote: post #14

Disclamer: probably dumb questions so, the 20b model. Can someone explain to me what I would need to do in terms of resources (GPU, I assume) if I want to run 20 concurrent processes, assuming I need 1k tokens/second throughput (on each, so 20 x 1k) Also, is this model better/comparable for information extraction compared to gpt-4.1-nano, and would it be cheaper to host myself 20b?

(answer for 1 inference) Al depends on the context length you want to support as the activation memory will dominate the requirements. For 4096 tokens you will get away with 24GB (or even 16GB), but if you want to go for the full 131072 tokens you are not going to get there with a 32GB consumer GPU like the 5090. You'll need to spring for at the minimum an A6000 (48GB) or preferably an RTX 6000 Pro (96GB). Also keep…

How Macs compare to RTXs for this? I.e. what numbers can be expected from Mac mini/Mac Studio with 64/128/256/512GB of unified memory?

Re: Open models by OpenAI

#225

Post by fnands »

fnands wrote: post #223

Mhh, I wonder if these are distilled from GPT4-Turbo. I asked it some questions and it seems to think it is based on GPT4-Turbo: > Thus we need to answer "I (ChatGPT) am based on GPT-4 Turbo; number of parameters not disclosed; GPT-4's number of parameters is also not publicly disclosed, but speculation suggests maybe around 1 trillion? Actually GPT-4 is likely larger than 175B; maybe 500B. In any case, we can note i…

Also:

> The user appears to think the model is "gpt-oss-120b", a new open source release by OpenAI. The user likely is misunderstanding: I'm ChatGPT, powered possibly by GPT-4 or GPT-4 Turbo as per OpenAI. In reality, there is no "gpt-oss-120b" open source release by OpenAI

Re: Open models by OpenAI

#226

Post by davio »

bogtog wrote: post #95
x187463 wrote: post #23

Running a model comparable to o3 on a 24GB Mac Mini is absolutely wild. Seems like yesterday the idea of running frontier (at the time) models locally or on a mobile device was 5+ years out. At this rate, we'll be running such models in the next phone cycle.

When people talk about running a (quantized) medium-sized model on a Mac Mini, what types of latency and throughput times are they talking about? Do they mean like 5 tokens per second or at an actually usable speed?

On a M1 MacBook Air with 8GB, I got this running Gemma 3n:

12.63 tok/sec • 860 tokens • 1.52s to first token

I'm amazed it works at all with such limited RAM

Re: Open models by OpenAI

#227

Post by anonymoushn »

Rhubarrbb wrote: post #220
n42 wrote: post #135

Earlier quoted context omitted.

here's a quick recording from the 20b model on my 128GB M4 Max MBP: https://asciinema.org/a/AiLDq7qPvgdAR1JuQhvZScMNr and the 120b: https://asciinema.org/a/B0q8tBl7IcgUorZsphQbbZsMM I am, um, floored

Generation is usually fast, but prompt processing is the main limitation with local agents. I also have a 128 GB M4 Max. How is the prompt processing on long prompts? processing the system prompt for Goose always takes quite a while for me. I haven't been able to download the 120B yet, but I'm looking to switch to either that or the GLM-4.5-Air for my main driver.

it's odd that the result of this processing cannot be cached.

Re: Open models by OpenAI

#228

Post by 01HNNWZ0MV43FF »

narrator wrote: post #159
4b6442477b1280b wrote: post #99

Earlier quoted context omitted.

in that era, OpenAI and Anthropic were still deluding themselves into thinking they would be the "stewards" of generative AI, and the last US administration was very keen on regoolating everything under the sun, so "safety" was just an angle for regulatory capture. God bless China.

Yeah, China is e/acc. Nice cheap solar panels too. Thanks China. The problem is their ominous policies like not allowing almost any immigration, and their domestic Han Supremacist propaganda, and all that make it look a bit like this might be Han Supremacy e/acc. Is it better than wester/decel? Hard to say, but at least the western/decel people are now starting to talk about building power plants, at least for datace…

I wish we had voter support for a hydrocarbon tax, though. It would level out the prices and then the AI companies can decide whether they want to pay double to burn pollutants or invest in solar and wind and batteries

Re: Open models by OpenAI

#229

Post by sabaimran »

Super excited to see these released!

Major points of interest for me:

- In the "Main capabilities evaluations" section, the 120b outperform o3-mini and approaches o4 on most evals. 20b model is also decent, passing o3-mini on one of the tasks.

- AIME 2025 is nearly saturated with large CoT

- CBRN threat levels kind of on par with other SOTA open source models. Plus, demonstrated good refusals even after adversarial fine tuning.

- Interesting to me how a lot of the safety benchmarking runs on trust, since methodology can't be published too openly due to counterparty risk.

Model cards with some of my annotations: https://openpaper.ai/paper/share/7137e6a8-b6ff-4293-a3ce-68b...

Re: Open models by OpenAI

#230

Post by boringg »

winterrx wrote: post #80
deviation wrote: post #13

So this confirms a best-in-class model release within the next few days? From a strategic perspective, I can't think of any reason they'd release this unless they were about to announce something which totally eclipses it?

GPT-5 coming Thursday.

How much hype do we anticipate with the release of GPT-5 or whichever name to be included? And how many new features?
Post reply on HN