Live data from Hacker News

Introducing System One Models and Jev

typesafe.ai

421–430 of 513 posts

Re: Introducing System One Models and Jev

#421

This is a good product but the naming/branding is pretty unfortunate. Typesafe.AI sounds like some typescript/structured output type of tool… What even is “system one” ? IMO the product/tech is really there, just needs better communication.

> What even is "system one"

I definitely agree it's underexplained in type safe.ai's materials.

I have to assume it's a reference to the fast, heuristic, intuitive "system 1" process in humans, as opposed to the slow, procedural, reasoning "system 2".

This theory is recognized, among others, in Daniel Kahneman 2002 Nobel prize on Economics.

Re: Introducing System One Models and Jev

#422

> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. Oh, I have one of those use cases, matching people in genealogy trees. You can ask all sorts of questions: do the names match? Do they match within some edit distance? Do they match according to soundex/ metaphone rules (which are themselves a ginormous set of r…

You need blocking!

Fundamentally this is an entity resolution problem. An LLM can score pairwise really well but scoring all the pairs would be insanely computationally difficult.

If you can constrain the set of potential matches up front by querying the dataset for things that could be matches it gets a lot more tractable to use an LLM for this.

Are there any heuristics you can use to reduce the search space? You mentioned soundex transformation and maybe prefixes of last names could work? Even if you get the number of potential matches down by a few orders of magnitude this gets more reasonable!

Check out https://moj-analytical-services.github.io/splink/index.html

Re: Introducing System One Models and Jev

#423
post #363

Earlier quoted context omitted.

There isn’t a single human in this world and hasn’t ever been that meets your happy-enough standard. Make of it what you will.

Why is that at all relevant? Humans are known to hallucinate a lot. Ask 10 different witnesses at a crime scene what they saw and they'll all report different things. A good, non-hallucinating LLM would only report things for which it has evidence. It would consult the facts every single time. It's a pain in the butt for humans to fact-check everything but LLMs can quickly look up all kinds of stuff. That's what make…

Yes, and for the LLM you can do it in multiple passes.

So you can bolt the fact-check / source-check pass onto whatever other system you have, without having to redesign the underlying system.

Re: Introducing System One Models and Jev

#429
post #218
post #100

Earlier quoted context omitted.

When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct

if it puts a high confidence value on a wrong answer, thats still hallucinating, no? llm hallucinations are high probability tokens that are incorrect vs the real world

What we would want to see if a confidence value that is in line with the actual correctness. If the value is 0.9 for 1000 different answers, then approximately 900 of those answers should be correct.

Re: Introducing System One Models and Jev

#430

Earlier quoted context omitted.

> I really have to say that I like their manifesto Their manifesto: "you only build on top of it if it's trustworthy." - the irony of this while putting out the most misleading, dishonest marketing campaign I've seen in months for their first public appearance doesn't exactly scream "trustworthy" to me.

What do you find dishonest?

Comments under the x post.
Post reply on HN