> [others] Output tokens: ~5x more expensive than input tokens. > [them] Output tokens: FREE (too cheap to meter). I'm very confused by this.
This is not confusing
Introducing System One Models and Jev
441–450 of 513 posts
Re: Introducing System One Models and Jev
#442Earlier quoted context omitted.
I'll tell you this. Output isn't too cheap to meter, there is no decoder.
So an encoder-only model with a classifier trained on the heads or something? DeepSeek recently switched to an encoder-decoder architecture in an attempt to get the best of both worlds (fast prefill while preserving generation capability), I wonder if that might be the future?
This is basically what they have. https://github.com/fastino-ai/GLiNER2
Re: Introducing System One Models and Jev
#443This is a very promising idea - a model that takes arbitrary text input (which can be a complex json), plus a set of questions (yes/no, multiple-choice, or score) and quickly (milliseconds) and cheaply ($0.042/MTok) answers those questions. Unfortunately, none of this is explained in the announcement, but the documentation [0] is pretty good. [0]: https://docs.typesafe.ai/concepts/how-to-build-with-system-o...
An example with manual combinatorial exclusion in “not_for” field made me cry, this is a wild hybrid of code logic, textual definitions, and AI blackbox. It’s a cool idea, but the “glue” layer is too boilerplate-ish
Re: Introducing System One Models and Jev
#444Earlier quoted context omitted.
Not to be tooo pedantic, but a bot that assigned 0 confidence to everything wouldn’t hallucinate. A calculator either gets the right answer or doesn’t answer. It wouldn’t have to be all knowing as long as it knew perfectly what it doesn’t know
A quantum calculator answers in distributions.
In one universe that is true, in another one not.
Re: Introducing System One Models and Jev
#445Wasn't really till seeing this home assistant demo they have ( https://www.loom.com/share/18c4dbcf8db546dfb2d7f2ef018e78e4 ) that the value really clicked for me. Seems really cool.
Thanks, much clearer that way. The speaker is unusually eloquent and to the point for a very nerdy demo, they should put that on their home page.
Re: Introducing System One Models and Jev
#446> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. Oh, I have one of those use cases, matching people in genealogy trees. You can ask all sorts of questions: do the names match? Do they match within some edit distance? Do they match according to soundex/ metaphone rules (which are themselves a ginormous set of r…
There's also two other important limitations to using an LLM and just providing it with pairs of records. It does not know enough about the records in the context of the overall dataset: - what is the data quality and to what extent do we expect a errors in some fields - how unusual are certain values such as names in the context of the dataset as a whole, e.g. some names would be very common in some countries but ra…
Another great article, thanks. I thought a lot about that too, and basically realized the same thing (this wasn't the coding agent), that frequency in the dataset mattered a lot for how useful a match is, and made frequency lists from my datasets, both on how common the literal names were and the phoneme-reduced names.
But I'm worried about that "bitter lesson" the TypeSafe CEO refers to, and that we're retreading the steps of natural language processing and a lot of other fields, trying to come up with clever rules, when the rule-based approach simply never gets good enough. I need matching to be good, it's absolutely central to genealogy, and I've seen the damage blindly linking by hand-crafted matching formulas can do.
Re: Introducing System One Models and Jev
#447So it's kind of like BERT but you don't have to train it for each request/response shape ?
Re: Introducing System One Models and Jev
#448Wasn't really till seeing this home assistant demo they have ( https://www.loom.com/share/18c4dbcf8db546dfb2d7f2ef018e78e4 ) that the value really clicked for me. Seems really cool.
That's good. Side note - just like most people don't need an intelligent personal assistant to manage and respond their emails and book their flights, most people also don't need smart homes. Century old toggle switches are more than enough in a 3 room apartment or 5 room house unless you have a mention.
I personally just find it convenient, even in my 3.5 room apartment.
Though I do have switches that work.
Re: Introducing System One Models and Jev
#449Wasn't really till seeing this home assistant demo they have ( https://www.loom.com/share/18c4dbcf8db546dfb2d7f2ef018e78e4 ) that the value really clicked for me. Seems really cool.
This is very cool. However I don’t really want to bounce all my home automation commands to the cloud. I hope there will be an open weights approach one day. I’ve spent a lot of time setting up my local only home automation system, it would suck if it didn’t work during an internet outage, and also there are obvious privacy problems.
Open weight or even closed with a proprietary license, I'd definitely like to be able to run it locally for all sorts of physical world stuff.
Re: Introducing System One Models and Jev
#450Earlier quoted context omitted.
There's also two other important limitations to using an LLM and just providing it with pairs of records. It does not know enough about the records in the context of the overall dataset: - what is the data quality and to what extent do we expect a errors in some fields - how unusual are certain values such as names in the context of the dataset as a whole, e.g. some names would be very common in some countries but ra…
Another great article, thanks. I thought a lot about that too, and basically realized the same thing (this wasn't the coding agent), that frequency in the dataset mattered a lot for how useful a match is, and made frequency lists from my datasets, both on how common the literal names were and the phoneme-reduced names. But I'm worried about that "bitter lesson" the TypeSafe CEO refers to, and that we're retreading th…
I agree, and I think LLMs can potentially do a better job than more conventional methods, so long as they're provided with enough content.
Another aspect of context that could be relevant to your work may be how people are nested within households. In some record linkage scenarios you can run a linkage on household membership and use this to make person linkages more precise. I imagine an LLM agent could also use this information effectively if it were explicitly provided, but sometimes they miss logical leaps like this