Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TypeChat

microsoft.github.io

Post reply on HN
Print view View original

101–110 of 174 posts

Re: TypeChat

#101

Post by msp26 »

LordDragonfang wrote: post #74
BoorishBears wrote: post #71

Earlier quoted context omitted.

Here's a project that does that better imo: https://github.com/dzhng/zod-gpt And by better I mean doesn't tie you to OpenAI for no good reason

I don't know where all you people work that your employer would prefer a random git repo (that has no support and no guarantee of updates) over a solution from Microsoft . (Alternatively: that you have so much free time that you'd prefer to fiddle with your own validation code instead of writing your actual app) Open source solutions are great (which this still is, btw), but having a first-party solution is also a go…

You're overrating the influence of the name Microsoft here. It's just some devs from the company working on this with no proper guarantee backing the project.

I've been through this whole song and dance already with Microsoft's Guidance (another LLM project) and could not justify using it further in production at work. We built some tools and wrappers ourselves and it wasn't even that difficult. These libraries are often more trouble than they're worth.

Re: TypeChat

#102

Post by just-ok »

akavi wrote: post #80
rezonant wrote: post #55

Earlier quoted context omitted.

For the TS compiler: If you took each generation step, closed any partial JSON objects (ie close any open `{`), checked that it was valid JSON and then validated it using a deep version of Partial , that should do the trick.

Not for even the simplest schemas. Eg, given even the type: {"aLongerKey": "value"} The generation prefix: {"a would by your algorithm produce the following invalid output: {"a}

But that isn’t valid JSON

Re: TypeChat

#103

Post by refulgentis »

ofslidingfeet wrote: post #96
verdverm wrote: post #89

Earlier quoted context omitted.

There are multiple ways to get structured output, and what this library is doing is not really that interesting. The concept is interesting and has had multiple implementations already, the code (and abstraction) here is not interesting and creates more issues than it solves

Tell me how to get reliably structured output. I'm all ears.

I have a prompt from February pre chatgpt and now I just use the models functions support, it's built for exactly that

Re: TypeChat

#104

Post by pizza »

verdverm wrote: post #87
whimsicalism wrote: post #51

Earlier quoted context omitted.

Yes as the abstractions gets better it becomes easier to code useful things.

the debate is about how valuable the abstraction here is to warrant a library, and the fact that it predefines the prompt and api call flow, so you cannot prompt engineer or use something like CoT/ToT

People can debate till the cows come home. But it's worth remembering that hacker news is about stimulating intellectual curiosity.

There's no reason for this to have a fixed flow, either - it's got a hint of diagonalizability to it - by which I mean, you can get the model to build a schema for dynamic flows, given a 'bootstrapping' schema. No different than what has always had to happen for someone to write a compiler for a programming language in the language itself.

Re: TypeChat

#105

Post by tlrobinson » 

    const schema = fs.readFileSync(path.join(__dirname, "sentimentSchema.ts"), "utf8");
    const translator = typechat.createJsonTranslator(model, schema, "SentimentResponse");
It would have been much nicer if they took this an an opportunity to build generic runtime type introspection into TypeScript.

Re: TypeChat

#106

Post by BoorishBears »

semiquaver wrote: post #97
BoorishBears wrote: post #71

Earlier quoted context omitted.

Here's a project that does that better imo: https://github.com/dzhng/zod-gpt And by better I mean doesn't tie you to OpenAI for no good reason

How does TypeChat tie you to OpenAI more than zod-gpt does? The interface required of a chat completion model is as simple as it gets, and you can provide your own easily (as the linked post makes clear) https://github.com/microsoft/TypeChat/blob/4d34a5005c67bc494...

The ergonomics of most of these AI libraries are built around using whatever models they provide integrations for: according to the file you linked retries won't even work unless you go and roll them in your implementation.

I'm sure someone will open a PR for Anthropic/Cohere/etc. but a quick glance made it pretty clear they made it with OpenAI-first in mind, or even low hanging fruit like retries would have been abstracted away at a higher level.

Re: TypeChat

#107

Post by huac »

I've written a version of this in Golang (tied to OpenAI API, mostly): https://github.com/stillmatic/gollum/blob/main/dispatch.go

Define a struct and tag it with golang's json comments. Then, give it a prompt and ...

    type dinnerParty struct {
        Topic       string   `json:"topic" jsonschema:"required" jsonschema_description:"The topic of the conversation"`
        RandomWords []string `json:"random_words" jsonschema:"required" jsonschema_description:"Random words to prime the conversation"`
    }
    completer := openai.NewClient(os.Getenv("OPENAI_API_KEY"))
    d := gollum.NewOpenAIDispatcher[dinnerParty]("dinner_party", "Given a topic, return random words", completer, nil)
    output, _ := d.Prompt(context.Background(), "Talk to me about dinosaurs")
and you should get a response like

    expected := dinnerParty{
        Topic:       "dinosaurs",
        RandomWords: []string{"dinosaur", "fossil", "extinct"},
    }

Re: TypeChat

#108

Post by huac »

behnamoh wrote: post #61
osaariki wrote: post #13

Earlier quoted context omitted.

I'm not familiar with how TypeChat works, but Guidance [1] is another similar project that can actually integrate into the token sampling to enforce formats. [1]: https://github.com/microsoft/guidance

except that guidance is defunct and is not maintained anymore.

did they announce that anywhere? it does appear like progress has slowed down quite a lot.

Re: TypeChat

#109

Post by icholy »

verdverm wrote: post #92
bwestergard wrote: post #50

Earlier quoted context omitted.

There are many subtleties to invoking the typescript type checker from node. It's nice to have support for that from the team that maintains the type checker.

Is the team working on typescript in a good position to be making LLM libraries, interfaces, and abstractions? Do they have the background and context to understand how their library fits into AI workflows? Could they have provided the same value with a blog post and sample code?

Your coworkers must love you.

Re: TypeChat

#110

Post by psyphy »

unshavedyak wrote: post #12
paxys wrote: post #3

I swear I think of something and Anders Hejlsberg builds it. Structured requests and responses are 100% the next evolution of LLMs. People are already getting tired of chatbots. Being able to plug in any backend without worrying about text parsing and prompts will be amazing.

> Structured requests and responses are 100% the next evolution of LLMs. People are already getting tired of chatbots. Being able to plug in any backend without worrying about text parsing and prompts will be amazing. Yup, a general desire of mine is to locally run an LLM which has actionable interfaces that i provide. Things like "check time", "check calendar", "send message to user" and etc. TypeChat seems to be in…

I just published CopilotKit, which lets you implement this exact functionality for any web app via react hooks.

`useMakeCopilotActionable` = you pass the type of the input, and an arbitrary typescript function implementation.

https://github.com/RecursivelyAI/CopilotKit

Feedback welcome

Post reply on HN