Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Olmo 3: Charting a path through the model flow to lead open-source AI

allenai.org

Post reply on HN
Print view View original

41–50 of 135 posts

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#41

Post by refulgentis »

embedding-shape wrote: post #36
tcsenpai wrote: post #17

I think they should start aiming for 20B models along with 32B and 7B. Usually 7B is enough for a 8GB GPU, 32B requires a 24GB GPU for decent quants (I can fit a 32B with IQ3_XXS but is not ideal) while 20-ish B models (such as magistral or gpt-oss) are a perfect fit for 16GB GPUs

Depends heavily on the architecture too, I think a free-for-all to find the better sizes is still kind of ongoing, and rightly so. GPT-OSS-120B for example fits in around 61GB VRAM for me when on MXFP4. Personally, I hope GPU makers instead start adding more VRAM, or if one can dream, expandable VRAM.

Unlikely to see more VRAM in the short term, memory prices are thru the roof :/ like, not subtly, 2-4x.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#42

Post by specproc »

specproc wrote: post #25

I don't know about you lot, but everything about the chart (Base Model Training / Post-Training) made my blood boil.

Downvoted, so I'll expand:

- The twin Y axes run to different ylims (50, 100 on left, 30, 80 on right). This is misleading in itself, as you're measuring two different things, but unforgivable when points are connected across the two subplots. This implies continuity in measurement, which does not appear to be the case.

- No x axis label, what is the time series supposed to represent? Is it even a time series?

- It doesn't even have a chuffing title, or the slightest or the slightest explanation of what it's supposed to represent.

Charts should explain and explore, not confuse and mislead. See me after class.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#43

Post by embedding-shape »

andy99 wrote: post #40
embedding-shape wrote: post #37

Earlier quoted context omitted.

Every time a new model is released, there are a bunch of reports or written experiences about people using the model with software that seemingly doesn't support it. GPT-OSS really made that clear, where 90% of the ecosystem declared it broken, but most people were using dumb quants and software that didn't properly support it. Guess we'll repeat the same thing with OLMo now.

There are a bunch (currently 3) of examples of people getting funny output, two of which saying it’s in LM studio (I don’t know what that is). It does seem likely that it’s somehow being misused here and the results aren’t representative.

Definitely. Usually I'd wait 2-3 weeks for the ecosystem to catch up and iron out the kinks, or do what I did for GPT-OSS, fix it in the places where it's broken, then judge it when I'm sure it's actually used correctly.

Otherwise, in that early period of time, only use the provided scripts/tools from the people releasing the model itself, which is probably the only way in those 2-3 weeks to be sure you're actually getting the expected responses.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#45

Post by embedding-shape »

refulgentis wrote: post #41
embedding-shape wrote: post #36

Earlier quoted context omitted.

Depends heavily on the architecture too, I think a free-for-all to find the better sizes is still kind of ongoing, and rightly so. GPT-OSS-120B for example fits in around 61GB VRAM for me when on MXFP4. Personally, I hope GPU makers instead start adding more VRAM, or if one can dream, expandable VRAM.

Unlikely to see more VRAM in the short term, memory prices are thru the roof :/ like, not subtly, 2-4x.

Well, GPUs are getting more VRAM, although it's pricey. But we didn't used to have 96GB VRAM GPUs at all, now they do exist :) But for the ones who can afford it, it is at least possible today. Slowly it increases.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#46

Post by dangoodmanUT »

What are some of the real world applications of small models like this, is it only on-device inference?

In most cases, I'm only seeing models like sonnet being just barely sufficiently for the workloads I've done historically. Would love to know where others are finding use of smaller models (like gpt-oss-120B and below, esp smaller models like this).

Maybe some really lightweight borderline-NLP classification tasks?

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#47

Post by mistrial9 »

Y_Y wrote: post #26

I asked it if giraffes were kosher to eat and it told me: > Giraffes are not kosher because they do not chew their cud, even though they have split hooves. Both requirements must be satisfied for an animal to be permissible. HN will have removed the extraneous emojis. This is at odds with my interpretation of giraffe anatomy and behaviour and of Talmudic law. Luckily old sycophant GPT5.1 agrees with me: > Yes. They h…

due to reforms around the first centuries of the Common Era, trivia questions to certain tribal priests are no longer a litmus test for acceptable public goods in the marketplace.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#48

Post by schopra909 »

dangoodmanUT wrote: post #46

What are some of the real world applications of small models like this, is it only on-device inference? In most cases, I'm only seeing models like sonnet being just barely sufficiently for the workloads I've done historically. Would love to know where others are finding use of smaller models (like gpt-oss-120B and below, esp smaller models like this). Maybe some really lightweight borderline-NLP classification tasks?

I think you nailed it.

For us it’s classifiers that we train for very specific domains.

You’d think it’d be better to just finetune a smaller non-LLM model, but empirically we find the LLM finetunes (like 7B) perform better.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#49

Post by [deleted] »

embedding-shape wrote: post #45
refulgentis wrote: post #41

Earlier quoted context omitted.

Unlikely to see more VRAM in the short term, memory prices are thru the roof :/ like, not subtly, 2-4x.

Well, GPUs are getting more VRAM, although it's pricey. But we didn't used to have 96GB VRAM GPUs at all, now they do exist :) But for the ones who can afford it, it is at least possible today. Slowly it increases.

[deleted]

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#50

Post by fnbr »

dangoodmanUT wrote: post #46

What are some of the real world applications of small models like this, is it only on-device inference? In most cases, I'm only seeing models like sonnet being just barely sufficiently for the workloads I've done historically. Would love to know where others are finding use of smaller models (like gpt-oss-120B and below, esp smaller models like this). Maybe some really lightweight borderline-NLP classification tasks?

(I’m a researcher on the post-training team at Ai2.)

7B models are mostly useful for local use on consumer GPUs. 32B could be used for a lot of applications. There’s a lot of companies using fine tuned Qwen 3 models that might want to switch to Olmo now that we have released a 32B base model.

Post reply on HN