Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing System One Models and Jev

typesafe.ai

Post reply on HN
Print view View original

271–280 of 512 posts

Re: Introducing System One Models and Jev

#271

Post by dozerly »

8note wrote: post #218
dbbk wrote: post #100

Earlier quoted context omitted.

When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct

if it puts a high confidence value on a wrong answer, thats still hallucinating, no? llm hallucinations are high probability tokens that are incorrect vs the real world

Yes, there is no magic sauce here that makes stochastic output binary if that’s what people are looking for.

Re: Introducing System One Models and Jev

#272

Post by cooljoseph »

zenlikethat wrote: post #231
cooljoseph wrote: post #213

A few questions: 1. Do you provide any kind of largest common subtree caching for cheaper input? 2. Have you tried auto-generating Lisp programs structurally? 3. Have you tried augmenting a Lisp language with a `choice` function that makes choices given a prompt, the environment, and the continuation stack?

(1) Nope, it's always the same input token cost (2-3) No, but that's kind of a sick cook ... Want to get access and try it? nathan@typesafe.ai

Thanks for the early access! I was testing the Lisp idea out in the playground, but I don't think the model is smart enough right now to generate actual code. I tried having Jev finish generating the code for a Fibonacci number function, but it kept wanting to create a literal number instead of refer to a variable which is a number. This happened both when I gave Jev the current program as a string and when I gave Jev the program as structured data.

Maybe I'm just not doing a very good job at prompting Jev, but I think right now it's not quite capable enough to generate Lisp code.

Link: https://console.typesafe.ai/playground?share=shr_148e1248984...

Re: Introducing System One Models and Jev

#273

Post by consumer451 »

Super cool! Instantly joined the waitlist.

It might be boring, but I can see exactly how I could use this right now to improve my agentic rag.[0] In two months I am supposed to deal with a giant corpus, while still maintaining responsive chat UX. I have been working my butt off to make our first big client happy. This could really help solve the chunk ranking problem.

[0] assuming the policies are compatible with sensitive production workloads, some time in the near future.

Re: Introducing System One Models and Jev

#275

Post by ramoz »

cfowles wrote: post #268

Wasn't really till seeing this home assistant demo they have ( https://www.loom.com/share/18c4dbcf8db546dfb2d7f2ef018e78e4 ) that the value really clicked for me. Seems really cool.

Guess I'm a bit less impressed seeing that for some of the more intelligent driven+action work -- splitting requests in the video -- they had to kick out to an anthropic model.

Re: Introducing System One Models and Jev

#276

Post by cooljoseph »

nickdonnelly wrote: post #252
riknos314 wrote: post #220

Earlier quoted context omitted.

This is likely still an LLM (in the purest definition of a language model with relatively many parameters) since the inputs are natural language, just not a generative LLM as the output is something other than more language.

The inputs aren't natural language. https://docs.typesafe.ai/primitives

The inputs are natural language, they're just also structured into a tree. The first example on that very page shows natural language instructions:

    questions = {
        "refund_requested": Noul(
            instructions="Does the customer request a refund?",
        ),
    }

Re: Introducing System One Models and Jev

#277

Post by resonious »

dozerly wrote: post #271
8note wrote: post #218

Earlier quoted context omitted.

if it puts a high confidence value on a wrong answer, thats still hallucinating, no? llm hallucinations are high probability tokens that are incorrect vs the real world

Yes, there is no magic sauce here that makes stochastic output binary if that’s what people are looking for.

Right and so maybe we should stop saying "can't hallucinate" when it can by definition.

Re: Introducing System One Models and Jev

#278

Post by rpunkfu »

dozerly wrote: post #271
8note wrote: post #218

Earlier quoted context omitted.

if it puts a high confidence value on a wrong answer, thats still hallucinating, no? llm hallucinations are high probability tokens that are incorrect vs the real world

Yes, there is no magic sauce here that makes stochastic output binary if that’s what people are looking for.

It’s not what people are looking for, but what they wrongly claim.

Re: Introducing System One Models and Jev

#279

Post by fgblanch »

cfowles wrote: post #268

Wasn't really till seeing this home assistant demo they have ( https://www.loom.com/share/18c4dbcf8db546dfb2d7f2ef018e78e4 ) that the value really clicked for me. Seems really cool.

This video makes a better job at explaining what it is about vs the marketing ones. Thanks for sharing.

Re: Introducing System One Models and Jev

#280

Post by iforgotmypasswo »

Could you use this to build a proactive memory formation and retrieval system for LLMs that runs lightning fast?

Last 32k of connect + Summary of current task: Did we learn something useful here (true/false)? What is the category to file it under? Then notify the LLM to file it away.

What class of memory might be useful here? Model gives probability to each item in the list. Short description of all memories ordered by tagged class is used in the next round. Are any of these memories useful in the current context, such that they will inform the model and help in its task (yes/no)?

I’m sure there’s some fine tuning to be had, but this sure seems like the basis for a substantially better proactive memory system that works around an existing LLM conversation.

If I’m understanding what this does and how this works (generic input, intelligent classification with probabilities, rapid and cheap), this is absolutely nuts.

Post reply on HN