[deleted]
TypeChat
21–30 of 174 posts
Re: TypeChat
#22How does no voice assistant (Apple, Google, Amazon, Microsoft) integrate LLMs into their service yet, and how has OpenAI not released their own voice assistant?
Also like RSS, if there were some standard URL a websites exposed for AI interaction, using this TypeChat to expose the interfaces, we'd be well on our way here.
Re: TypeChat
#23Someone should just get this working on Llama 2 instead of O̶p̶e̶n̶AI.com [0] All this is it's just talking to a AI model sitting on someone else's server. [0] https://github.com/microsoft/TypeChat/blob/main/src/model.ts...
The most recent gpt4all
(https://github.com/nomic-ai/gpt4all)
includes a local server compatible with OpenAPI -- this could be a useful start!
Re: TypeChat
#24How does no voice assistant (Apple, Google, Amazon, Microsoft) integrate LLMs into their service yet, and how has OpenAI not released their own voice assistant? Also like RSS, if there were some standard URL a websites exposed for AI interaction, using this TypeChat to expose the interfaces, we'd be well on our way here.
Microsoft is doing that to replace Cortana in windows 11
Re: TypeChat
#25How does no voice assistant (Apple, Google, Amazon, Microsoft) integrate LLMs into their service yet, and how has OpenAI not released their own voice assistant? Also like RSS, if there were some standard URL a websites exposed for AI interaction, using this TypeChat to expose the interfaces, we'd be well on our way here.
Willow, and the Willow Interference Server have the option to use Vicuna with speech input and TTS
Re: TypeChat
#26Looks like it just runs the LLM in a loop until it spits out something that type checks, prompting with the error message. This is a cute idea and it looks like it should work, but I could see this getting expensive with larger models and input prompts. Probably not a fix for all scenarios.
I suspect most products are concerned about product-market fit then they can wrangle costs down.
There's also a good assumption that models will be improving structured output as the market is demanding it.
Re: TypeChat
#27I don't see the value add here.
Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat...
You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as models and user needs evolve.
As an example, think about how you could change the prompt or use python classes instead. How much work would this be using a library like this versus something that lifts the API calls and text templating to the user like: https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/llm...
Re: TypeChat
#28It's not clear to me how they ensure the responses will be valid JSON, are they just asking for it, then parsing the result with error checking?
seems like they run the generated response through the typescript type checker, and if it fails, retry using the error message as a further hint to the LLM, until it succeeds.
I would expect that, if it doesn’t do that even, why bother… that is also trivial to do anyway.
Re: TypeChat
#29I swear I think of something and Anders Hejlsberg builds it. Structured requests and responses are 100% the next evolution of LLMs. People are already getting tired of chatbots. Being able to plug in any backend without worrying about text parsing and prompts will be amazing.
> Structured requests and responses are 100% the next evolution of LLMs. People are already getting tired of chatbots. Being able to plug in any backend without worrying about text parsing and prompts will be amazing. Yup, a general desire of mine is to locally run an LLM which has actionable interfaces that i provide. Things like "check time", "check calendar", "send message to user" and etc. TypeChat seems to be in…
I was thinking about this yesterday. ChatGPT really is good enough to act as a proper virtual assistant / home manager, with enough toggles exposed.
Re: TypeChat
#30It's not super clear how this differs from another recently released library from Microsoft: Guidance (https://github.com/microsoft/guidance).
They both seem to aim to solve the problem of getting typed, valid responses back from LLMs