41–50 of 174 posts
So, it's a thing that appends "please format your response as the following JSON" to the prompt", then validates the actual response against the schema, all in a "while (true)" loop (literally) until it succeeds. This unbelievable achievement is a work of seven people (authors of the blog post). Honestly, this is getting beyond embarrassing. How is this the world we live in?
I don't see the value add here. Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat... You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as mod…
The value is in: 1. Running the typescript type checker against what is returned by the LLM. 2. If there are type errors, combining those into a "repair prompt" that will (it is assumed) have a higher likelihood of eliciting an LLM output that type checks. 3. Gracefully handling the cases where the heuristic in #2 fails. https://github.com/microsoft/TypeChat/blob/main/src/typechat... In my experience experimenting wi…
Why would I want to add all this extra stuff just for that? The opaque retry until it returns valid JSON? That sounds like it will make for many pleasant support cases or issues
Personally, I have found investing more effort in the actual prompt engineering improves success rates and reduces the need to retry with an appended error message. Especially helpful are input/output pairs (i.e. few-shot) and while we haven't tried it yet, I imagine fine-tuning and distillation would improve the situation even more
So, it's a thing that appends "please format your response as the following JSON" to the prompt", then validates the actual response against the schema, all in a "while (true)" loop (literally) until it succeeds. This unbelievable achievement is a work of seven people (authors of the blog post). Honestly, this is getting beyond embarrassing. How is this the world we live in?
I think the (arguably very prototypical) implementation is not what's interesting here. It's the concept itself. Natural language may soon become the default interface for most of the computing people do on a day to day basis, and tools like these will make it easier to create new applications in this space.
- Lots of examples / prompt engineering techniques
- MS Guideance
- TypeChat
- OpenAI functions (the model itself is tuned to do this, a key differentiator)
- ...others?
So, it's a thing that appends "please format your response as the following JSON" to the prompt", then validates the actual response against the schema, all in a "while (true)" loop (literally) until it succeeds. This unbelievable achievement is a work of seven people (authors of the blog post). Honestly, this is getting beyond embarrassing. How is this the world we live in?
One of the authors is Anders Hejlsberg, the guy behind c# and delphi
How does no voice assistant (Apple, Google, Amazon, Microsoft) integrate LLMs into their service yet, and how has OpenAI not released their own voice assistant? Also like RSS, if there were some standard URL a websites exposed for AI interaction, using this TypeChat to expose the interfaces, we'd be well on our way here.
Seriously, it feels like there’s some collusion going on behind the scenes. This is the most obvious use case for the technology, but none of the big vendors have explored it.
Earlier quoted context omitted.
seems like they run the generated response through the typescript type checker, and if it fails, retry using the error message as a further hint to the LLM, until it succeeds.
I would expect that, if it doesn’t do that even, why bother… that is also trivial to do anyway.
It's not clear to me how they ensure the responses will be valid JSON, are they just asking for it, then parsing the result with error checking?
Earlier quoted context omitted.
The value is in: 1. Running the typescript type checker against what is returned by the LLM. 2. If there are type errors, combining those into a "repair prompt" that will (it is assumed) have a higher likelihood of eliciting an LLM output that type checks. 3. Gracefully handling the cases where the heuristic in #2 fails. https://github.com/microsoft/TypeChat/blob/main/src/typechat... In my experience experimenting wi…
these are trivial steps you can add in any script, as your link demonstrates. Why would I want to add all this extra stuff just for that? The opaque retry until it returns valid JSON? That sounds like it will make for many pleasant support cases or issues Personally, I have found investing more effort in the actual prompt engineering improves success rates and reduces the need to retry with an appended error message.…