Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TypeChat

microsoft.github.io

Post reply on HN
Print view View original

61–70 of 174 posts

Re: TypeChat

#61

Post by behnamoh »

osaariki wrote: post #13
33a wrote: post #11

Looks like it just runs the LLM in a loop until it spits out something that type checks, prompting with the error message. This is a cute idea and it looks like it should work, but I could see this getting expensive with larger models and input prompts. Probably not a fix for all scenarios.

I'm not familiar with how TypeChat works, but Guidance [1] is another similar project that can actually integrate into the token sampling to enforce formats. [1]: https://github.com/microsoft/guidance

except that guidance is defunct and is not maintained anymore.

Re: TypeChat

#62

Post by J_Shelby_J »

9dev wrote: post #37
sdwr wrote: post #29

Earlier quoted context omitted.

I was thinking about this yesterday. ChatGPT really is good enough to act as a proper virtual assistant / home manager, with enough toggles exposed.

ChatGPT isn’t the limiting factor here, a good way to expose the toggles is. I recently tried to expose our company CRM to employees by means of a Teams bot they could ask for stuff in natural language (like „send an invite link to newlead@example.org“ or „how many MAUs did customer Foo have in June“), but while I almost got there, communicating an ever-growing set of actionable commands (with an arbitrary number of…

https://github.com/ShelbyJenkins/LLM-OpenAPI-minifier

I have a working solution to exposing the toggles.

I’m integrating it into the bot I have in the other repo.

Goal is you point to an openapi spec and then GPT can run choose and run functions. Basically Siri but with access to any API.

Re: TypeChat

#63

Post by DanRosenwasser »

Zaheer wrote: post #30

It's not super clear how this differs from another recently released library from Microsoft: Guidance ( https://github.com/microsoft/guidance ). They both seem to aim to solve the problem of getting typed, valid responses back from LLMs

One of the key things that we've focused on with TypeChat is not just that it acts as a specification for retrieving structured data (i.e. JSON), but that the structure is actually valid - that it's well-typed based on your type definitions.

The thing to keep in mind with these different libraries is that they are not necessarily perfect substitutes for each other. They often serve different use-cases, or can be combined in various ways -- possibly using the techniques directly and independent of the libraries themselves.

Re: TypeChat

#64

Post by J_Shelby_J »

osaariki wrote: post #13
33a wrote: post #11

Looks like it just runs the LLM in a loop until it spits out something that type checks, prompting with the error message. This is a cute idea and it looks like it should work, but I could see this getting expensive with larger models and input prompts. Probably not a fix for all scenarios.

I'm not familiar with how TypeChat works, but Guidance [1] is another similar project that can actually integrate into the token sampling to enforce formats. [1]: https://github.com/microsoft/guidance

It’s logit bias. You don’t even need another library to do this. You can do it with three lines of python.

Here’s an example of one of my implementations of logit bias.

https://github.com/ShelbyJenkins/shelby-as-a-service/blob/74...

Re: TypeChat

#65

Post by robbie-c »

This is funny, I have something pretty similar in my code, except it's using Zod for runtime typechecking, and I convert Zod schemas to json schemas and send that to gpt-3.5 as a function call. I would expect that using TypeScript's output is better for recovering from errors than with Zod's output, so I can definitely see the advantage of this.

Re: TypeChat

#67

Post by sidnb13 »

paxys wrote: post #3

I swear I think of something and Anders Hejlsberg builds it. Structured requests and responses are 100% the next evolution of LLMs. People are already getting tired of chatbots. Being able to plug in any backend without worrying about text parsing and prompts will be amazing.

Maybe worth looking into: https://news.ycombinator.com/item?id=36750083

Re: TypeChat

#68

Post by sidnb13 »

paxys wrote: post #3

I swear I think of something and Anders Hejlsberg builds it. Structured requests and responses are 100% the next evolution of LLMs. People are already getting tired of chatbots. Being able to plug in any backend without worrying about text parsing and prompts will be amazing.

maybe worth looking into: https://news.ycombinator.com/item?id=36750083

Re: TypeChat

#69

Post by nfw2 »

verdverm wrote: post #27

I don't see the value add here. Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat... You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as mod…

It’s essentially prompt engineering as a service with some basic quality-control features thrown in.

Sure, your engineers could implement it themselves, but don’t they have better things to do?

Re: TypeChat

#70

Post by ofslidingfeet »

verdverm wrote: post #27

I don't see the value add here. Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat... You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as mod…

Getting these models to reliably return a consistent structure without frequent human intervention and/or having to account for the personal moral opinions of big tech CEOs is not trivial, no.
Post reply on HN