my sons name is also Jev
Introducing System One Models and Jev
291–300 of 513 posts
Re: Introducing System One Models and Jev
#292Earlier quoted context omitted.
Technically speaking when you send the prefix “The capital of France is “ into an LLM it will also produce probabilities across its whole vocabulary.
The probability values don’t really represent confidence in modern LLMs though, especially after RLHF and RLVR. System One says they use RLCD, Reinforcement Learning for Calibrated Decisions, which presumably has accurate probabilities as an explicit optimisation goal.
How is that different from RLVR?
Re: Introducing System One Models and Jev
#293Earlier quoted context omitted.
Yeah but what stops it from producing confidently incorrect outputs...
Nothing, but imagine using LLMs for a classification task People out there are so resigned to the models being unreliable that they are really doing things like hallucinating deliberately, and then matching the hallucinations to embeddings - https://softwaredoug.com/blog/2026/08/10/hypothetical-classi... You could do that or you could just... use a model that will never produce unreliable outputs in the first place.
I'm certainly not resigned to that, at least for classification.
Even non-frontier models are absurdly good at this in a broad sense.
Which would make it hard to judge "a model that will never produce unreliable outputs in the first place" against something that is already really, really good and exceptional in domain-specific areas with the tiniest amount of elbow grease.
Speed and cost look good though (for now)!
Re: Introducing System One Models and Jev
#294Can HN have a tag for open-weight vs closed-source models please? The progress is nice, but if it is not released at least in papers or open-weight? These are just ads?
Re: Introducing System One Models and Jev
#295If I understand correctly, it can play chess and rubic cube better than LLM ? ( may be go too ? )
ok I've searched, it may not, but it can do "driving car" and "trade 0-DTE Option" much better.
Re: Introducing System One Models and Jev
#296Earlier quoted context omitted.
His claim was that the title is misleading, not sure how it's relevant to that claim that you use "string models" (full LLMs). The original title before it changed less than an hour ago was: "Jev: New frontier model 40-400x cheaper and 20-200x faster" I'm going to agree that was misleading. And on the second point: >>Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a co…
> Type safety is not factual correctness. I very much agree with this and want to hone in on where do actually disagree. Would you say a linear classifier hallucinates?
> I very much agree with this
From the intro blog
"Hallucination and type-safety are intrinsically related"
I'm not entirely sure why we're conflating type safety with, I guess, value or output safety.
"Would you say a linear classifier hallucinates?"
No, but it can be (and often is) mathematically correct and functionally incorrect. It doesn't help to say "a linear classifier can't hallucinate" when you get even 99% accuracy. That's 100% a semantic play, and it doesn't help when the picture of a dog is labeled cat and the response is "yeah but that's not a hallucination, only stupid LLMs do that"
Re: Introducing System One Models and Jev
#297This sounds good but so far all claims just sound like marketing terms. I'd love to see real proof. e.g. "RLCD" and "parallel sampling" have nothing to back it up. also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster. Nonetheless i want this to be…
love that you love the manifesto! letting the first batches off the waitlist now, but we do have some early users describing their experience ( https://x.com/danshipper/status/2099947471518474522 )
Congrats ! Really excited for the team.
Re: Introducing System One Models and Jev
#298Earlier quoted context omitted.
I would guess a tiny stripped down text diffusion model. It only has 32k context, and for choice mode it can only select from 10 choices.
> and for choice mode it can only select from 10 choices. Rip there goes my excitement. I have a task that something like this would be great for but the list of options is a zero or two larger than that xd
you can still chain them
Re: Introducing System One Models and Jev
#299Earlier quoted context omitted.
The probability values don’t really represent confidence in modern LLMs though, especially after RLHF and RLVR. System One says they use RLCD, Reinforcement Learning for Calibrated Decisions, which presumably has accurate probabilities as an explicit optimisation goal.
How is that different from RLVR?
RLVR generally upweights tokens along the whole thinking trace that led to a correct answer, whether each token was "correct" or not. RLVR doesn't train a model to output an 80% likelihood, it just trains it to produce correct answers, and not to produce incorrect ones.
System One hasn't said how RLCD works, but they do say it is explicitly training models to output "calibrated" probabilities, which makes it distinct from RLVR. This is how they describe it:
> System One models are trained for calibrated decisions: their probabilities are optimized against outcomes to reflect uncertainty.
Re: Introducing System One Models and Jev
#300First, congrats to the team on launching something genuinely interesting and new. Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that. This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do. Jev can only generate structured out…
When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct
I read "hallucinations" as "generates novel output with no grounding/source". i.e. "it just made something completely up".
I believe their "accuracy" metric (sonnet 5 level) is where "right/wrong" is measured.