I think I missed why is this faster? What I’m reading here is it’s similar to constrained decoding but I’m not seeing the explanation of why it’s able to get those results.
Introducing System One Models and Jev
301–310 of 513 posts
Re: Introducing System One Models and Jev
#302Earlier quoted context omitted.
Guess I'm a bit less impressed seeing that for some of the more intelligent driven+action work -- splitting requests in the video -- they had to kick out to an anthropic model.
Haiku, to rewrite a sentence as two discreet commands. I agree that it was notable that they delegated to an existing LLM, but I don't think it detracts much from the value proposition (not yet proven) of their demo.
[deleted]
Re: Introducing System One Models and Jev
#303Wasn't really till seeing this home assistant demo they have ( https://www.loom.com/share/18c4dbcf8db546dfb2d7f2ef018e78e4 ) that the value really clicked for me. Seems really cool.
This is very cool. However I don’t really want to bounce all my home automation commands to the cloud. I hope there will be an open weights approach one day. I’ve spent a lot of time setting up my local only home automation system, it would suck if it didn’t work during an internet outage, and also there are obvious privacy problems.
Re: Introducing System One Models and Jev
#304Earlier quoted context omitted.
When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct
if it puts a high confidence value on a wrong answer, thats still hallucinating, no? llm hallucinations are high probability tokens that are incorrect vs the real world
Correct, they have not made a universal all-knowing omniscient oracle, which is what would be required for "can't hallucinate".
Re: Introducing System One Models and Jev
#305[deleted]
Re: Introducing System One Models and Jev
#306Wasn't really till seeing this home assistant demo they have ( https://www.loom.com/share/18c4dbcf8db546dfb2d7f2ef018e78e4 ) that the value really clicked for me. Seems really cool.
This is very cool. However I don’t really want to bounce all my home automation commands to the cloud. I hope there will be an open weights approach one day. I’ve spent a lot of time setting up my local only home automation system, it would suck if it didn’t work during an internet outage, and also there are obvious privacy problems.
GLiClass is performant, and its zero-shot classification scores are in the same ballpark as the Terra-level results Jev points to.
Re: Introducing System One Models and Jev
#307First, congrats to the team on launching something genuinely interesting and new. Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that. This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do. Jev can only generate structured out…
When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct
What about the LLM calls though that are done midchain? In the Home Assistant video the multi-intent prompt gets split using what looks like a traditional llm model, which I'm assuming is vulnerable to classical hallucinations.
Re: Introducing System One Models and Jev
#308Earlier quoted context omitted.
This is very cool. However I don’t really want to bounce all my home automation commands to the cloud. I hope there will be an open weights approach one day. I’ve spent a lot of time setting up my local only home automation system, it would suck if it didn’t work during an internet outage, and also there are obvious privacy problems.
GLiClass is performant, and its zero-shot classification scores are in the same ballpark as the Terra-level results Jev points to. https://github.com/knowledgator/gliclass
Thanks!
Re: Introducing System One Models and Jev
#309Seems like "some" of LLMs tasks are now Jev tasks.
Re: Introducing System One Models and Jev
#310It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing. It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence). Edit: On the AI primer pa…
CEO here - that is right! I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable). But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)
It's unclear if the context extends as the conversation grows?