31–40 of 174 posts
It's not clear to me how they ensure the responses will be valid JSON, are they just asking for it, then parsing the result with error checking?
seems like they run the generated response through the typescript type checker, and if it fails, retry using the error message as a further hint to the LLM, until it succeeds.
I don't see the value add here. Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat... You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as mod…
But that said it still feels like using a library is the right thing to do... so I'm still watching this space to see what matures and emerges as a good-enough approach.
Hang on, so this is doing runtime validation of an object against a typescript type definition? Can this be shipped as a standalone library/feature? This would be absolutely game changing for validating api response payloads, etc. in typescript codebases.
https://github.com/microsoft/TypeChat/blob/4d34a5005c67bc494...
Why this instead of GPT Functions?
In the end, both methods try to coax the model into returning a JSON object, one method can be used with any model, the other is tied to a specific, ever changing vendor API
Why would one choose to only support "OpenAI" and nothing else?
Hang on, so this is doing runtime validation of an object against a typescript type definition? Can this be shipped as a standalone library/feature? This would be absolutely game changing for validating api response payloads, etc. in typescript codebases.
maybe this function? https://github.com/microsoft/TypeChat/blob/4d34a5005c67bc494...
It's interesting because I've always been under the impression the TS team was against the use of types at runtime (that's why projects like https://github.com/nonara/ts-patch exist), but now they're doing it themselves with this project...
I wonder what the performance overhead of starting up an instance of tsc in memory is? Is this suitable for low latency situations? Lots of testing to do...
Earlier quoted context omitted.
> Structured requests and responses are 100% the next evolution of LLMs. People are already getting tired of chatbots. Being able to plug in any backend without worrying about text parsing and prompts will be amazing. Yup, a general desire of mine is to locally run an LLM which has actionable interfaces that i provide. Things like "check time", "check calendar", "send message to user" and etc. TypeChat seems to be in…
I was thinking about this yesterday. ChatGPT really is good enough to act as a proper virtual assistant / home manager, with enough toggles exposed.
How does no voice assistant (Apple, Google, Amazon, Microsoft) integrate LLMs into their service yet, and how has OpenAI not released their own voice assistant? Also like RSS, if there were some standard URL a websites exposed for AI interaction, using this TypeChat to expose the interfaces, we'd be well on our way here.
Honestly, this is getting beyond embarrassing. How is this the world we live in?
I don't see the value add here. Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat... You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as mod…
1. Running the typescript type checker against what is returned by the LLM.
2. If there are type errors, combining those into a "repair prompt" that will (it is assumed) have a higher likelihood of eliciting an LLM output that type checks.
3. Gracefully handling the cases where the heuristic in #2 fails.
https://github.com/microsoft/TypeChat/blob/main/src/typechat...
In my experience experimenting with the same basic idea, the heuristic in #2 works surprisingly well for relatively simple types (i.e. records and arrays not nested too deeply, limited use of type variables). It turns out that prompting LLMs to return values inhabiting relatively simple types can be used to create useful applications. Since that is valuable, this library is valuable inasmuch as it eliminates the need to hand roll this request pattern, and provides a standardized integration with the typescript codebase.