Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Olmo 3: Charting a path through the model flow to lead open-source AI

allenai.org

Post reply on HN
Print view View original

61–70 of 135 posts

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#61

Post by turnsout »

mentalgear wrote: post #29

This is how the future of "AI" has to look like: Fully-traceable inferences steps, that can be inspected & adjusted if needed. Without this, I don't see how we (the general population) can maintain any control - or even understanding - of these larger and more opaque becoming LLM-based long-inference "AI" systems. Without transparency, Big Tech, autocrats and eventually the "AI" itself (whether "self-aware" or not) w…

I agree transparency is great. But making the response inspectable and adjustable is a huge UI/UX challenge. It's good to see people take a stab at it. I hope there's a lot more iteration in this area, because there's still a long way to go.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#63

Post by embedding-shape »

refulgentis wrote: post #57
embedding-shape wrote: post #45

Earlier quoted context omitted.

Well, GPUs are getting more VRAM, although it's pricey. But we didn't used to have 96GB VRAM GPUs at all, now they do exist :) But for the ones who can afford it, it is at least possible today. Slowly it increases.

Agreed, in the limit, RAM go up. As billg knows, 128KB definitely wasn't enough for everyone :)

I'm already thinking 96GB might not be enough, and I've only had this GPU for 6 months or so :|

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#64

Post by littlestymaar »

fnbr wrote: post #50
dangoodmanUT wrote: post #46

What are some of the real world applications of small models like this, is it only on-device inference? In most cases, I'm only seeing models like sonnet being just barely sufficiently for the workloads I've done historically. Would love to know where others are finding use of smaller models (like gpt-oss-120B and below, esp smaller models like this). Maybe some really lightweight borderline-NLP classification tasks?

(I’m a researcher on the post-training team at Ai2.) 7B models are mostly useful for local use on consumer GPUs. 32B could be used for a lot of applications. There’s a lot of companies using fine tuned Qwen 3 models that might want to switch to Olmo now that we have released a 32B base model.

May I ask why you went for a 7B and a 32B dense models instead of a small MoE like Qwen3-30B-A3B or gpt-oss-20b given how successful these MoE experiments were?

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#65

Post by lionkor »

turnsout wrote: post #61
mentalgear wrote: post #29

This is how the future of "AI" has to look like: Fully-traceable inferences steps, that can be inspected & adjusted if needed. Without this, I don't see how we (the general population) can maintain any control - or even understanding - of these larger and more opaque becoming LLM-based long-inference "AI" systems. Without transparency, Big Tech, autocrats and eventually the "AI" itself (whether "self-aware" or not) w…

I agree transparency is great. But making the response inspectable and adjustable is a huge UI/UX challenge. It's good to see people take a stab at it. I hope there's a lot more iteration in this area, because there's still a long way to go.

If I give you tens of billions of dollars, like, wired to your personal bank account, do you think you could figure it out given a decade or two?

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#66

Post by oivey »

zamadatix wrote: post #58
latexr wrote: post #51

Earlier quoted context omitted.

> How many times did you retry (so it's not just up to chance) If you don’t know the answer to a question, retrying multiple times only serves to amplify your bias, you have no basis to know the answer is correct.

If you retry until it gives the answer you want then it only serves to amplify your bias. If you retry and see how often it agrees with itself then it serves to show there is no confidence in an answer all around. It's a bit of a crutch for LLMs lacking the ability to just say "I'm not sure" because doing so is against how they are rewarded in training.

You’re still likely to just amplify your own bias if you don’t do some basic experimental controls like having some preselected criteria on how many retries you’re going to do or how many agreeing trials are statistically significant.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#67

Post by moffkalast »

schopra909 wrote: post #48
dangoodmanUT wrote: post #46

What are some of the real world applications of small models like this, is it only on-device inference? In most cases, I'm only seeing models like sonnet being just barely sufficiently for the workloads I've done historically. Would love to know where others are finding use of smaller models (like gpt-oss-120B and below, esp smaller models like this). Maybe some really lightweight borderline-NLP classification tasks?

I think you nailed it. For us it’s classifiers that we train for very specific domains. You’d think it’d be better to just finetune a smaller non-LLM model, but empirically we find the LLM finetunes (like 7B) perform better.

I think it's no surprise that any model that has a more general understanding of text performs better than some tiny ad-hoc classifier that blindly learns a couple of patterns and has no clue what it's looking at. It's going to fail in much weirder ways that make no sense, like old cnn-based vision models.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#68

Post by moffkalast »

mentalgear wrote: post #29

This is how the future of "AI" has to look like: Fully-traceable inferences steps, that can be inspected & adjusted if needed. Without this, I don't see how we (the general population) can maintain any control - or even understanding - of these larger and more opaque becoming LLM-based long-inference "AI" systems. Without transparency, Big Tech, autocrats and eventually the "AI" itself (whether "self-aware" or not) w…

You've answered your own question as to why many people will want this approach gone entirely.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#70

Post by comp_raccoon »

silviot wrote: post #21

I tried the playground at https://playground.allenai.org/ and clicked the "Show OlmoTrace" button. Above the response it says > Documents from the training data that have exact text matches with the model response. Powered by infini-gram so, if I understand correctly, it searches the training data for matches in the LLM output. This is not traceability in my opinion. This is an attempt at guessing. Checking individua…

Olmo researcher here. The point of OlmoTrace is not no attribute the entire response to one document in the training data—that’s not how language models “acquire” knowledge, and finding a single or few documents as support for an answer is impossible.

The point of OlmoTrace is to show that fragments of model response are influenced by its training data. sometimes is how specific adjectives are used together in way that seem unnatural to us, but are combination of training data (ask for a movie review!)

A favorite example of mine is asking to tell a joke or ask for a random number, because strangely all LLMs return the same joke or number. Well with OlmoTrace, you can see which docs in the training data contain the super common response!

hope this helps

Post reply on HN