Introducing System One Models and Jev • Hacker News Board

Re: Introducing System One Models and Jev

#281

Post by vopi » Wed, Sep 16, 2026, 2:22 AM UTC

This is actually pretty cool. I think the undertalked about part of this for TypeSafe is that they can always "extract"/distill the frontier of this type of task from the newest LLMs for cheap. Jev seems seems to be GPT-6-Astra/Fable 5.1 but I imagine a bunch of training data is from earlier models?

Then, you can serve it faster/cheaper than the frontier LLMs. It's basically distilling a small but extremely common use-case from LLMs and serving it. Then RLCD comes into play to update weights when a new model comes out, etc.

Any thoughts on what the next potential "cheap" win to be distilled from frontier LLMs is? I'm going to need to play around with this.

Re: Introducing System One Models and Jev

#282

Post by dozerly » Wed, Sep 16, 2026, 2:25 AM UTC

copperx wrote: post #199

CompleteSkeptic wrote: post #124

Earlier quoted context omitted.

we hope so! the bigger hope is to not just eat LLM market share, but to allow for people to use AI much more in the inner loop of software

I'm sure you've thought of self-driving. How does the model work in that space?

This smells like a tool a more broadly capable LLM would take advantage of extremely well.

Re: Introducing System One Models and Jev

#283

Post by 10c8 » Wed, Sep 16, 2026, 2:26 AM UTC

Wow, this is really cool. If this holds up to scrutiny, and has a decent context window (+16k), it suddenly changes our project's status from "cool concept, too slow and expensive to release" to "doable", just like that.

Just joined the waitlist, excited to try it out!

Re: Introducing System One Models and Jev

#284

Post by kzsh » Wed, Sep 16, 2026, 2:30 AM UTC

ramoz wrote: post #275

cfowles wrote: post #268

Wasn't really till seeing this home assistant demo they have ( https://www.loom.com/share/18c4dbcf8db546dfb2d7f2ef018e78e4 ) that the value really clicked for me. Seems really cool.

Guess I'm a bit less impressed seeing that for some of the more intelligent driven+action work -- splitting requests in the video -- they had to kick out to an anthropic model.

Haiku, to rewrite a sentence as two discreet commands.

I agree that it was notable that they delegated to an existing LLM, but I don't think it detracts much from the value proposition (not yet proven) of their demo.

Re: Introducing System One Models and Jev

#285

Post by cfowles » Wed, Sep 16, 2026, 2:30 AM UTC

ramoz wrote: post #275

cfowles wrote: post #268

Wasn't really till seeing this home assistant demo they have ( https://www.loom.com/share/18c4dbcf8db546dfb2d7f2ef018e78e4 ) that the value really clicked for me. Seems really cool.

Guess I'm a bit less impressed seeing that for some of the more intelligent driven+action work -- splitting requests in the video -- they had to kick out to an anthropic model.

That's fair, but it highlights how this would actually be used. It doesn't really seem like a competitor to other models but instead a way to make these real systems more enjoyable to deal with.

Re: Introducing System One Models and Jev

#286

Post by cooljoseph » Wed, Sep 16, 2026, 2:32 AM UTC

adroitboss wrote: post #61

I am positive I know exactly how this works, I made something similar a few months back. But the problem is without generation you are extremely limited in the use cases. And while the model can't hallucinate, it can still be wrong. It just can't make up data.

Last year I also had a rather similar idea, but dropped it before I went very far in working on it. I wonder if you and I had similar ideas?

1. Start with an LLM, so that your model understands natural language.

2. Replace RoPE with a tree embedding scheme, and causal attention with a sparse attention on the graph structure. (You could use full attention... but it's cheaper to use graph attention.)

3. Chop off the final unembedding layer, replacing it with a projection down to two scalars, one for logits and one for confidence.

4. Each option of a choice is represented by a number of tokens in leaf position; average these tokens' logit outputs to get the option's logit. Average all of the confidences from all of the options to get the choice's confidence.

5. Train the logits by KL divergence from a true distribution (or NLL on samples from a true distribution).

6. Train the confidences on a subset of the data in which you know the entire true distribution.

The hardest part is getting real world data for workflows, but I wildly speculate that you can get by with only ~50,000 documents if you first adapt domains using synthetic data.

Re: Introducing System One Models and Jev

#287

Post by [deleted] » Wed, Sep 16, 2026, 2:36 AM UTC

[deleted]

Re: Introducing System One Models and Jev

#288

Post by wesammikhail » Wed, Sep 16, 2026, 2:42 AM UTC

bigglebear wrote: post #258

alphazard wrote: post #170

There's a whole lot of information on this page that doesn't tell me anything about what this actually is. Can anyone spell out what the architecture is here? They claim it's not an LLM, which I read as "not an auto-regressive token generator". I assume they are still using a transformer, otherwise they would be talking about the thing that's not a transformer, instead of all the fluff on the linked page. But they em…

I would guess a tiny stripped down text diffusion model. It only has 32k context, and for choice mode it can only select from 10 choices.

> and for choice mode it can only select from 10 choices.

Rip there goes my excitement. I have a task that something like this would be great for but the list of options is a zero or two larger than that xd

Re: Introducing System One Models and Jev

#289

Post by nojvek » Wed, Sep 16, 2026, 2:47 AM UTC

CompleteSkeptic wrote: post #236

nightshift1 wrote: post #233

The whole page reads like it was vibe-written by an AI. If I'd built something as disruptive as this claims to be, I'd have spent at least fifteen minutes writing the announcement myself. Every time I see 'we' in an announcement like this, I picture one guy alone in his basement.

unfortunately all hand-written :( my chief-of-staff does unironically handwrite em dashes though

I really appreciated the hand-written release. Thank you.

Re: Introducing System One Models and Jev

#290

Post by iforgotmypasswo » Wed, Sep 16, 2026, 2:47 AM UTC

skerit wrote: post #41

So in theory you could feed it incomplete text, and then ask it for the probabilities of what the next character could be?

Or a partially completed song, asking for the next note. I’m not sure if you’re joking, but using it for space constrained next token generation within a grammar sounds like a really neat use case.