Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing System One Models and Jev

typesafe.ai

Post reply on HN
Print view View original

141–150 of 512 posts

Re: Introducing System One Models and Jev

#141

Post by CompleteSkeptic »

WhitneyLand wrote: post #136
CompleteSkeptic wrote: post #110

Earlier quoted context omitted.

I'm biased but I wouldn't call it misleading - generating text is super awesome and flexible, (we describe that in the blog post - and I personally use string models all the time) but it's true you pay a high tax for autoregressive generation > Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. that is likely true of all ML! perhaps we coul…

His claim was that the title is misleading, not sure how it's relevant to that claim that you use "string models" (full LLMs). The original title before it changed less than an hour ago was: "Jev: New frontier model 40-400x cheaper and 20-200x faster" I'm going to agree that was misleading. And on the second point: >>Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a co…

> Type safety is not factual correctness.

I very much agree with this and want to hone in on where do actually disagree. Would you say a linear classifier hallucinates?

Re: Introducing System One Models and Jev

#142

Post by speedping »

whazor wrote: post #138

A question I have, with the type { output: string }, would the model not become a LLM? And if it does, shouldn’t it cost the same as a LLM for output?

I don’t see this as an option in their website

You could theoretically ask “what is the next appropriate character?” and add the entire ascii charset but i doubt it’d work well and you’d be implementing autoregressive churn across network latency…

Re: Introducing System One Models and Jev

#143

Post by tensegrist »

what is the…epistemic status, for lack of a better way to put it, of the probabilities? what do they mean? what (probabilistic) guarantees do we have about, say, the responses to

- is the capital of france paris?

- it is august. is it raining in paris?

(forgive the examples; they're probably not semantically the sort of thing jev is trained to work on. but i figure the point translates to various kinds of questions that come up in "inner loop of agentic pid controller" contexts)

a normal text-generating model if asked to produce a number will also do that just fine. i assume in jev's case it was actually rled to essentially learn to express priors over things using its implicit world model, which definitely ought to help, but can we say more?

Re: Introducing System One Models and Jev

#144

Post by CompleteSkeptic »

mckngbrd wrote: post #106
CompleteSkeptic wrote: post #65

Earlier quoted context omitted.

CEO here - that is right! I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable). But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)

here is how I attempted to explain it to my company's AI group chat, is this roughly accurate? "instead of autoregressive string output it instead outputs structured type-safe 'decisions' with probabilities/confidence scores, each generated in parallel so sort of more like a Large Classification Model than a Large Language Model? or, maybe better to think of it as a sort of "shift left" in the LLM's transformer archi…

very accurate!

the one nuance I'd get into is I'd call it "zero-shot" over "instruction-tuned" (the latter often implies a particular distribution), but very safe for sharing

Re: Introducing System One Models and Jev

#145

Post by CompleteSkeptic »

ianbutler wrote: post #99
CompleteSkeptic wrote: post #65

Earlier quoted context omitted.

CEO here - that is right! I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable). But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)

I see this super interestingly as the "subconscious" to the llms "conscious" for lack of better terms. I'm super interested in this for broad and rapid decision making in the context of consumer agents so will be signing up for sure.

1. I am extremely on the same page 2. I do think that subconscious is not only much smarter than we give it credit for, but also much more robust than the "jagged frontier" of current LLMs

(shilling my blog post on that jaggedness: https://www.completeskeptic.com/p/lies-damned-lies-and-bench...)

Re: Introducing System One Models and Jev

#146

Post by CompleteSkeptic »

mushufasa wrote: post #18

I would love for things like this to be accessible via hubs like open router or AWS bedrock. It's hard to justify adding new model vendors directly with all the heightened concerns about privacy and security, but if bold new capabilities are added to a centralized already-vendor like AWS, technical people can adopt them without going through a whole compliance/purchasing/vendor review process. And an extra middleman…

They don't like adding stealth startups :(

Re: Introducing System One Models and Jev

#147

Post by elcomet »

The technology and the results are very handwavy. What is RLCD exactly ? What are scores on benchmarks compared to LLMs ?

This website does not inspire confidence at all, it all sounds like a marketing piece. I wish it was true, some kind of text-prompted classifier with LLM performance would be cool, but I can't trust it with what we are given.

Re: Introducing System One Models and Jev

#148

Post by CompleteSkeptic »

whazor wrote: post #138

A question I have, with the type { output: string }, would the model not become a LLM? And if it does, shouldn’t it cost the same as a LLM for output?

strings (and all sequential data structures) are not allowed at all - this is how we make sure all outputs can be computed in parallel (thus no output token cost)

Re: Introducing System One Models and Jev

#149

Post by WhitneyLand »

janalsncm wrote: post #113
jacobgold wrote: post #81

First, congrats to the team on launching something genuinely interesting and new. Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that. This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do. Jev can only generate structured out…

I don’t think it’s misleading if you compare on the use cases they suggested. It’s faster and cheaper (no idea if higher quality), so it’s immediately interesting for certain things. And if you buy their RLCD claims, this might be even better than huge models that know a bunch of irrelevant things.

What was misleading was the original title:

"Jev: New frontier model 40-400x cheaper and 20-200x faster"

I'm not the gatekeeper of who gets to call themselves a frontier model, but I don't think most people would count Jev in that group. It sounds false.

If their specific claims hold up, then it would make more sense to say something like:

"Advanced the speed/cost frontier for structured decisions"

Post reply on HN