Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing System One Models and Jev

typesafe.ai

Post reply on HN
Print view View original

101–110 of 512 posts

Re: Introducing System One Models and Jev

#101

Post by [deleted] »

CompleteSkeptic wrote: post #65
big_toast wrote: post #52

It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing. It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence). Edit: On the AI primer pa…

CEO here - that is right! I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable). But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)

[deleted]

Re: Introducing System One Models and Jev

#102

Post by Flere-Imsaho »

CompleteSkeptic wrote: post #65
big_toast wrote: post #52

It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing. It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence). Edit: On the AI primer pa…

CEO here - that is right! I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable). But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)

Hi - first congratulations, System One looks really promising.

The Doom demo really help me, at least, to understand how System One differs from LLMs. However the first demo (Side-by-side demonstration) - I'm struggling to understand what is going on here!

Re: Introducing System One Models and Jev

#103

Post by big_toast »

zenlikethat wrote: post #66
big_toast wrote: post #52

It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing. It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence). Edit: On the AI primer pa…

> the model takes as input a state (structured text? not sure if multi-modal) Input, and criteria/instructions can both be defined as structured input (JSON). This ends up being pretty powerful because the model is trained to understand structure. e.g.: https://docs.typesafe.ai/primitives/advanced#structured-inst... > not sure if multi-modal just JSON... for now :) > outputs the question's answers as appropriate corr…

I assume this isn't really for consumers/individuals currently? Kinda feels like an improved magic 8 ball.

I can't really intuit how I should think about when the model will be accurate. Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.

Re: Introducing System One Models and Jev

#104

Post by wxw »

> Input tokens: $0.042 / MTok ($42 per billion tokens).

> Output tokens: FREE (too cheap to meter).

Insane. The video demos are really compelling, in particular the speed.

> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems.

I buy this vision. A lot of LLM integration I see these days is ultimately exactly this. OpenAI-style structured outputs works decently but this would be a great improvement in cost, latency.

Re: Introducing System One Models and Jev

#105

Post by CompleteSkeptic »

petesergeant wrote: post #80

This is basically a zero-shot classifier that can accept raw text (or structured text) as an input, and is able to classify that text as accurately (they claim) as a frontier-level LLM. I have workflows this would be useful for, looking forward to it showing up on OpenRouter.

exactly right!

Re: Introducing System One Models and Jev

#106

Post by mckngbrd »

CompleteSkeptic wrote: post #65
big_toast wrote: post #52

It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing. It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence). Edit: On the AI primer pa…

CEO here - that is right! I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable). But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)

here is how I attempted to explain it to my company's AI group chat, is this roughly accurate?

"instead of autoregressive string output it instead outputs structured type-safe 'decisions' with probabilities/confidence scores, each generated in parallel

so sort of more like a Large Classification Model than a Large Language Model? or, maybe better to think of it as a sort of "shift left" in the LLM's transformer architecture, allowing you to replace the predefined token vocabulary of an LLM with a prescribed set of 'decisions' that need to be made based off the input context; and exposing those probabilities directly so they can be integrated into the system logic, instead of just sampling from top-K.

all of this while still being instruction-tuned (!!!)"

It's always been possible to build classification pipelines using LLM embeddings as the input. seems like this is a much more sophisticated / useful application of that concept

Re: Introducing System One Models and Jev

#108

Post by ttul »

CompleteSkeptic wrote: post #65
big_toast wrote: post #52

It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing. It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence). Edit: On the AI primer pa…

CEO here - that is right! I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable). But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)

For many day-to-day computing use cases, Jev seems far better suited than an autoregressive language model, if for no other reason than it is not wasting compute thinking about anything other than how to spit out a decision.

Do you have an architectural explainer yet for Jev or are you holding that close to your chest and letting the magic rip for now?

Re: Introducing System One Models and Jev

#110

Post by CompleteSkeptic »

jacobgold wrote: post #81

First, congrats to the team on launching something genuinely interesting and new. Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that. This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do. Jev can only generate structured out…

I'm biased but I wouldn't call it misleading - generating text is super awesome and flexible, (we describe that in the blog post - and I personally use string models all the time) but it's true you pay a high tax for autoregressive generation

> Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.

that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do

Post reply on HN