181–190 of 512 posts
I like the idea of System one models but all LLMs so far work as system 1 thinking because humans generate speech subconsciously with system 1.
System 2 thinking requires consciousness which AI does not have, so even reasoning models are still system 1 thinking as system 1 in humans has reasoning with heuristics.
Its limited but most people navigate the world with it completely, so it's enough for AI.
This is a good product but the naming/branding is pretty unfortunate. Typesafe.AI sounds like some typescript/structured output type of tool… What even is “system one” ? IMO the product/tech is really there, just needs better communication.
I mean, it's a structured output model that (apparently) can't hallucinate. I don't mind the name.
I think that this specific part is not super interesting if your harness just recovers from invalid LLM outputs.
The latency and cost - yes, those are super interesting.
Earlier quoted context omitted.
I don’t think it’s misleading if you compare on the use cases they suggested. It’s faster and cheaper (no idea if higher quality), so it’s immediately interesting for certain things. And if you buy their RLCD claims, this might be even better than huge models that know a bunch of irrelevant things.
What was misleading was the original title: "Jev: New frontier model 40-400x cheaper and 20-200x faster" I'm not the gatekeeper of who gets to call themselves a frontier model, but I don't think most people would count Jev in that group. It sounds false. If their specific claims hold up, then it would make more sense to say something like: "Advanced the speed/cost frontier for structured decisions"
Hasn't there been a lot talk about Astra's opaque reasoning capabilities (being able to think through complex questions without using a chain of thought)? Given that, can't you just replicate Jev by telling Astra "here is the question, you must make a multiple choice decision / output a score between 1-10, please answer directly in a single word, no reasoning allowed"? (Edit: Ok, Jev is much cheaper in input tokens s…
we actually use astra (and fable) in this way for our evals: evals.typesafe.ai
someone on the team cooked hard on that and it shows example traces comparing our model to opus/sol
This is a good product but the naming/branding is pretty unfortunate. Typesafe.AI sounds like some typescript/structured output type of tool… What even is “system one” ? IMO the product/tech is really there, just needs better communication.
We already had encoder models that skipped text generation for giving us a numerical output that could be computed as a probability. we also got no hallucinations and faster inference for free there. So we already had
1. "unstructured state in, probabilistic decisions out" 2. "orders of magnitude faster and more efficient"
What was hard there was to train the model head without ML expertise, and considerable amount of data.
This seems like this is a democratization of those encoders? The addition over existing encoders seems to be coming from being able to specify the output shape (up to a cardinality of 255). It is unclear to me if this is possible using Jev without additional labels for fine-tuning.
If so, that is still very impressive, but I think the faster inference and 0 hallucinations might come for free, from it not being generative.
Yes, I think it's a mistake that many organizations are cramming LLMs inside of automated pipelines where the extreme generality/flexibility of the model is at odds with the fact that you're using it for a very specific task that gets repeated over and over, and needs a very specific structured output to be successful. But specifying your task carefully (as well as deciding what counts as your input state representation etc) seems like a form of programming. Something (a person or a model working in a relatively unrestricted way) will need to produce a configuration/specification for this system.
So rather than Jev vs Claude I imagine that using Claude/ChatGPT/whatever interactively to define / refine your Jev config which then runs in prod might be the happy combination?
Earlier quoted context omitted.
When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct
Technically speaking when you send the prefix “The capital of France is “ into an LLM it will also produce probabilities across its whole vocabulary.