Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Olmo 3: Charting a path through the model flow to lead open-source AI

allenai.org

Post reply on HN
Print view View original

121–130 of 135 posts

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#121

Post by anon373839 »

refulgentis wrote: post #97
embedding-shape wrote: post #63

Earlier quoted context omitted.

I'm already thinking 96GB might not be enough, and I've only had this GPU for 6 months or so :|

Hehe me too…went all out on a MBP in 2022, did it again in April. Only upgrade I didn’t bother with was topping out at 128 GB of RAM instead of 64. Then, GPT-OSS 120B comes out and quickly makes me very sad I can’t use it locally

Same. I repeatedly kick myself for not getting the 128GB version, although not for the GPT-OSS model because I really haven’t been too impressed with it (through cloud providers). But now it’s best to wait until the M5 Max is out due to the new GPU neural accelerators that should greatly speed up prompt processing.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#122

Post by Sabinus »

embedding-shape wrote: post #37
tcsenpai wrote: post #20

Also: I simply asked to the 7B model "hi, who are u": Thought for 1 seconds Okay, the user is asking "Who are u?" with a casual tone ("u" instead of "you", exclamation mark). Let me break this down. First, I need to address the question itself. They want to know who I am as an AI assistant. My standard response is that I'm DeepSeek R1, designed to be helpful and handle reasoning step by step. But maybe they're not fa…

Every time a new model is released, there are a bunch of reports or written experiences about people using the model with software that seemingly doesn't support it. GPT-OSS really made that clear, where 90% of the ecosystem declared it broken, but most people were using dumb quants and software that didn't properly support it. Guess we'll repeat the same thing with OLMo now.

I'm really glad to read this, as this was my experience in LM studio with olmo. Worked for the first message but got progressively more unstable. Also doesn't seem to reset model state for a new conversation, every response following the model load gets progressively worse, even in new chats.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#123

Post by marvinalone »

marvinalone wrote: post #92
pardizeri wrote: post #30

Loaded olmo-3-7b-think in LM Studio 0.3.31 I asked: What language were you trained in? Is there a language you prefer for asking questions?" The thinking part: Okay, I need to make this statement true: "all the good guys win" is part of a puzzle. Okay? Return to work. Clever Hans was one of the most famous paradoxes that has ever been posed about the hard problem of consciousness, and your task is to argue for the ex…

Hmm. We think there is something wrong with the LM Studio integration. Thinking traces can be whacky, but not like this. We're looking into it.

The LM Studio team is on the case! Can you share settings you used to make that reply?

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#124

Post by thot_experiment »

thot_experiment wrote: post #111
andai wrote: post #60

Earlier quoted context omitted.

I'm out of the loop... so Qwen3-30B-VL is smart and Qwen3-30B is dumb... and that has to do not with the size but architecture?

ahaha sorry that was unclear, while i think the VL version is maybe a bit more performant, by "dumb" i meant any low quant low size model you're going to run locally, vs a "smart" model in my book is something like Opus 4.1 or Gemma 3. I basically class LLM queries into two categories, there's stuff i expect most models to get, and there's stuff i expect only the smartest models to have a shot of getting right, there…

sorry i meant gemini 3

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#125

Post by Imustaskforhelp »

moffkalast wrote: post #68
mentalgear wrote: post #29

This is how the future of "AI" has to look like: Fully-traceable inferences steps, that can be inspected & adjusted if needed. Without this, I don't see how we (the general population) can maintain any control - or even understanding - of these larger and more opaque becoming LLM-based long-inference "AI" systems. Without transparency, Big Tech, autocrats and eventually the "AI" itself (whether "self-aware" or not) w…

You've answered your own question as to why many people will want this approach gone entirely.

I always really like answers like yours as they are clever and in my opinion maybe a bit true as well

I think that tho there are a lot of things public can do and maybe raising awareness about these stuff could be great as well.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#126

Post by trebligdivad »

fnbr wrote: post #76
littlestymaar wrote: post #64

Earlier quoted context omitted.

May I ask why you went for a 7B and a 32B dense models instead of a small MoE like Qwen3-30B-A3B or gpt-oss-20b given how successful these MoE experiments were?

MoEs have a lot of technical complexity and aren't well supported in the open source world. We plan to release a MoE soon(ish). I do think that MoEs are clearly the future. I think we will release more MoEs moving forward once we have the tech in place to do so efficiently. For all use cases except local usage, I think that MoEs are clearly superior to dense models.

Even local, MoE are just so much faster, and they let you pick a large/less quantized model and still get a useful speed.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#127

Post by [deleted] »

Flere-Imsaho wrote: post #81
Y_Y wrote: post #26

I asked it if giraffes were kosher to eat and it told me: > Giraffes are not kosher because they do not chew their cud, even though they have split hooves. Both requirements must be satisfied for an animal to be permissible. HN will have removed the extraneous emojis. This is at odds with my interpretation of giraffe anatomy and behaviour and of Talmudic law. Luckily old sycophant GPT5.1 agrees with me: > Yes. They h…

Models should not have memorised whether animals are kosher to eat or not. This is information that should be retrieved from RAG or whatever. If a model responded with "I don't know the answer to that", then that would be far more useful. Is anyone actually working on models that are trained to admit not knowing an answer to everything?

[deleted]

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#128

Post by kurthr »

fnbr wrote: post #50
dangoodmanUT wrote: post #46

What are some of the real world applications of small models like this, is it only on-device inference? In most cases, I'm only seeing models like sonnet being just barely sufficiently for the workloads I've done historically. Would love to know where others are finding use of smaller models (like gpt-oss-120B and below, esp smaller models like this). Maybe some really lightweight borderline-NLP classification tasks?

(I’m a researcher on the post-training team at Ai2.) 7B models are mostly useful for local use on consumer GPUs. 32B could be used for a lot of applications. There’s a lot of companies using fine tuned Qwen 3 models that might want to switch to Olmo now that we have released a 32B base model.

Are there quantized (eg 4bit) models available yet? I assume the training was done in BF16, but it seems like most inference models are distributed in BF8 until they're quantized.

edit ahh I see it on huggingface: https://huggingface.co/mlx-community/Olmo-3-1125-32B-4bit

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#129

Post by nickreese »

grosswait wrote: post #119
nickreese wrote: post #103

Earlier quoted context omitted.

I run lmstudio for ease of use on several mac studios that are fronted by a small token aware router that estimates resource usage on the mac studios. Lots of optimization left there, but the systems are pinned most of the time so not focused on that at the moment as the gpus are the issue not the queuing.

I would like to hear more about your set up if you’re willing. Is the token aware router you’re using publicly available or something you’ve written yourself?

It isn't open... but drop me an email and I can send you it. Basically just tracks a list of known lmstudios on the network, queries their models every 15 seconds and routes to the ones who have the requested models loaded in a FIFO queue tracking the number of tokens/model (my servers are uniform... m4 max 128gb studios but could also track the server) and routes to the one that has just finished. I used to have it queue one just as it was expected to finish but was facing timeout issues due to an edgecase.
Post reply on HN