Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Olmo 3: Charting a path through the model flow to lead open-source AI

allenai.org

Post reply on HN
Print view View original

111–120 of 135 posts

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#111

Post by thot_experiment »

andai wrote: post #60
thot_experiment wrote: post #11

Qwen3-30B-VL is going to be fucking hard to beat as a daily driver, it's so good for the base 80% of tasks I want an AI for, and holy fuck is it fast. 90tok/s on my machine, I pretty much keep it in vram permanently. I think this sort of work is important and I'm really glad it's being done, but in terms of something I want to use every day there's no way a dense model can compete unless it's smart as fuck. Even dumb…

I'm out of the loop... so Qwen3-30B-VL is smart and Qwen3-30B is dumb... and that has to do not with the size but architecture?

ahaha sorry that was unclear, while i think the VL version is maybe a bit more performant, by "dumb" i meant any low quant low size model you're going to run locally, vs a "smart" model in my book is something like Opus 4.1 or Gemma 3.

I basically class LLM queries into two categories, there's stuff i expect most models to get, and there's stuff i expect only the smartest models to have a shot of getting right, there's some stuff in the middle ground that a quant model running locally might not get but something dumb but acceptable like Sonnet 4.5 or Kimi K2 might be able to handle.

I generally just stick to the two extremes and route my queries accordingly. I've been burned by sonnet 4.5/gpt-5 too many times to trust it.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#112

Post by thot_experiment »

solarkraft wrote: post #31
a96 wrote: post #27

Earlier quoted context omitted.

I'm only superficially familiar with these, but curious. Your comment above mentioned the VL model. Isn't that a different model or is there an a3b with vision? Would it be better to have both if I'd like vision or does the vision model have the same abilities as the text models?

Looks like it: https://ollama.com/library/qwen3-vl:30b-a3b

fwiw on my machine it is 1.5x faster to inference in llama.cpp, these the settings i use for inference for the qwen i just keep in vram permanently

    llama-server --host 0.0.0.0 --model Qwen3-VL-30B-A3B-Instruct-UD-Q4_K_XL.gguf --mmproj qwen3-VL-mmproj-F16.gguf --port 8080 --jinja --temp 0.7 --top-k 20 --top-p 0.8 -ngl 99 -c 65536 --repeat_penalty 1.0 --presence_penalty 1.5

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#113

Post by anonym29 »

Flere-Imsaho wrote: post #81
Y_Y wrote: post #26

I asked it if giraffes were kosher to eat and it told me: > Giraffes are not kosher because they do not chew their cud, even though they have split hooves. Both requirements must be satisfied for an animal to be permissible. HN will have removed the extraneous emojis. This is at odds with my interpretation of giraffe anatomy and behaviour and of Talmudic law. Luckily old sycophant GPT5.1 agrees with me: > Yes. They h…

Models should not have memorised whether animals are kosher to eat or not. This is information that should be retrieved from RAG or whatever. If a model responded with "I don't know the answer to that", then that would be far more useful. Is anyone actually working on models that are trained to admit not knowing an answer to everything?

>Models should not have memorised whether animals are kosher to eat or not.

Agreed. Humans do not perform rote memorization for all possibilities of rules-based classifications like "kosher or not kosher".

>This is information that should be retrieved from RAG or whatever.

Firm disagreement here. An intelligent model should either know (general model) or RAG-retrieve (non-general model) the criteria for evaluating whether an animal is kosher or not, and infer based on knowledge of the animal (either general model, or RAG-retrieval for a non-general model) whether or not the animal matches the criteria.

>If a model responded with "I don't know the answer to that", then that would be far more useful.

Again, firm disagreement here. "I don't know" is not a useful answer to a question that can be easily answered by cross-referencing easily-verifiable animal properties against the classification rules. At the very least, an intelligent model should explain which piece of information it is missing (properties of the animal in question OR the details of the classification rules), rather than returning a zero-value response.

To wit: if you were conducting an interview for a developer candidate, and you asked them whether Python supports functions, methods, both, or neither, would "I don't know" ever be an appropriate answer, even if the candidate genuinely didn't know off the top of their head? Of course not - you'd desire a candidate who didn't know to say something more along the lines of "I don't know, but here's what I would do to figure out the answer for you".

A plain and simple "I don't know" adds zero value to the conversation. While it doesn't necessarily add negative value to the conversation the way a confidently incorrect answer does, the goal for intelligent models should never be to produce zero value, it should be to produce nonzero positive value, even when it lacks required information.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#114

Post by thot_experiment »

dangoodmanUT wrote: post #46

What are some of the real world applications of small models like this, is it only on-device inference? In most cases, I'm only seeing models like sonnet being just barely sufficiently for the workloads I've done historically. Would love to know where others are finding use of smaller models (like gpt-oss-120B and below, esp smaller models like this). Maybe some really lightweight borderline-NLP classification tasks?

I have Qwen3-30B-VL (an MoE model) resident in my VRAM at all times now because it is quicker to use it to answer most basic google questions. The type of stuff like remembering how to force kill a WSL instance which i don't do that often is now frictionless because i can just write on terminal (q is my utility)

    q how to force kill particular WSL
and it will respond with "wsl --terminate " much faster than google

it's also quite good at tool calling, if you give it shell access it'll happily do things like "find me files over 10mb modified in the last day" etc where remembering the flags and command structure if you're not doing that action regularly previously required a google or a peek at the manpage

i also use it to transcribe todo lists and notes and put them in my todo app as well as text manipulation, for example if i have a list of like, API keys and URLs or whatever that i need to populate into a template, I can just select the relevant part of the template in VSCode, put the relevant data in the context and say "fill this out" and it does it faster than i would be able to do the select - copy - select - paste loop, even with my hard won VIM knowledge

TL;DR

It's very fast (90tok/s) and very low latency and that means it can perform a lot of mildly complex tasks that have an obvious solution faster than you.

and fwiw i don't even think sonnet 4.5 is very useful, it's a decent model but it's very common for me to push it into a situation where it will be subtly wrong and waste a lot of my time (of course that's colored by it being slow and costs money)

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#115

Post by robrenaud »

Open source AI is just a lost term. It has been co-opted. If the weights are released, it's open source. Not because that makes sense, not because it's right, but because that's the unfortunate marketting term that has stuck.

The real open source models need a rebrand. Transparent models, perhaps?

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#116

Post by robrenaud »

Flere-Imsaho wrote: post #81
Y_Y wrote: post #26

I asked it if giraffes were kosher to eat and it told me: > Giraffes are not kosher because they do not chew their cud, even though they have split hooves. Both requirements must be satisfied for an animal to be permissible. HN will have removed the extraneous emojis. This is at odds with my interpretation of giraffe anatomy and behaviour and of Talmudic law. Luckily old sycophant GPT5.1 agrees with me: > Yes. They h…

Models should not have memorised whether animals are kosher to eat or not. This is information that should be retrieved from RAG or whatever. If a model responded with "I don't know the answer to that", then that would be far more useful. Is anyone actually working on models that are trained to admit not knowing an answer to everything?

Benchmarks need to change.

There is a 4 choice choice question. Your best guess is the answer is B, at about 35% chance of being right. If you are graded on fraction of questions answered correctedly, the optimization pressure is simply to answer B.

If you could get half credit for answering "I don't know", we'd have a lot more models saying that when they are not confident.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#117

Post by khimaros »

Oras wrote: post #107
andy99 wrote: post #106

Earlier quoted context omitted.

Isn’t this before any curation has happened? I looked at it, I can see why it looks bad, but if they’re really being open about the whole pipeline, they have to include everything. Giving them a hard time for it only promotes keeping models closed. That said I like to think of it was my dataset I would have shuffled that part down in the list so it didn’t show up on the hf preview

Hard time? What value does adult videos description, views and comments add to small (7,32B) models?

what if that's where they learned how to utilize the double entendre? hard times indeed.

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#119

Post by grosswait »

nickreese wrote: post #103
alach11 wrote: post #91

Earlier quoted context omitted.

Just curious - are you using Open WebUI or Librechat as a local frontend or are all your workflows just calling the models directly without UI?

I run lmstudio for ease of use on several mac studios that are fronted by a small token aware router that estimates resource usage on the mac studios. Lots of optimization left there, but the systems are pinned most of the time so not focused on that at the moment as the gpus are the issue not the queuing.

I would like to hear more about your set up if you’re willing. Is the token aware router you’re using publicly available or something you’ve written yourself?

Re: Olmo 3: Charting a path through the model flow to lead open-source AI

#120

Post by Sabinus »

marvinalone wrote: post #93
RagnarD wrote: post #38

Fired it up on LM Studio using my RTX 6000 Pro Max-Q. I'm wondering why it thinks that it was developed by OpenAI. I may be missing it but I don't see any mention that AllenAI is starting with an open OpenAI model. > Tell me about yourself allenai/olmo-3-32b-think Thought for 6.33 seconds I'm an AI language model developed by OpenAI, designed to assist with a wide range of tasks such as answering questions, generatin…

We think there is something you with LM Studio's version of Olmo. We're investigating. Sorry about this.

Oh good, because mine would perform very poorly when the conversation got past 1 turn, or if I start a new chat without completely reloading the model.
Post reply on HN