Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TypeChat

microsoft.github.io

Post reply on HN
Print view View original

71–80 of 174 posts

Re: TypeChat

#71

Post by BoorishBears »

bwestergard wrote: post #40
verdverm wrote: post #27

I don't see the value add here. Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat... You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as mod…

The value is in: 1. Running the typescript type checker against what is returned by the LLM. 2. If there are type errors, combining those into a "repair prompt" that will (it is assumed) have a higher likelihood of eliciting an LLM output that type checks. 3. Gracefully handling the cases where the heuristic in #2 fails. https://github.com/microsoft/TypeChat/blob/main/src/typechat... In my experience experimenting wi…

Here's a project that does that better imo:

https://github.com/dzhng/zod-gpt

And by better I mean doesn't tie you to OpenAI for no good reason

Re: TypeChat

#72

Post by dvt »

This is my hot take: we're slowly entering the "tooling" phase of AI, where people realize there's no real value generation here, but people are so heavily invested in AI, that money is still being pumped into building stuff (and of course, it's one of the best way to guarantee your academic paper gets published). I mean, LangChain is kind of a joke and they raised $10M seed lol.

DeFi/crypto went through this phase 2 years ago. Mark my words, it's going to end up being this weird limbo for a few years where people will slowly realize that AI is a feature, not a product. And that its applicability is limited and that it won't save the world. It won't be able to self-drive cars due to all the edge cases, it won't be able to perform surgeries because it might kill people, etc.

I keep mentioning that even the most useful AI tools (Copilot, etc.) are marginally useful at best. At the very best it saves me a few clicks on Google, but the agents are not "intelligent" in the least. We went through a similar bubble a few years ago with chatbots[1]. These days, no one cares about them. "The metaverse" was much more short-lived, but the same herd mentality applies. "It's the next big thing" until it isn't.

[1] https://venturebeat.com/business/facebook-opens-its-messenge...

Re: TypeChat

#73

Post by unshavedyak »

9dev wrote: post #37
sdwr wrote: post #29

Earlier quoted context omitted.

I was thinking about this yesterday. ChatGPT really is good enough to act as a proper virtual assistant / home manager, with enough toggles exposed.

ChatGPT isn’t the limiting factor here, a good way to expose the toggles is. I recently tried to expose our company CRM to employees by means of a Teams bot they could ask for stuff in natural language (like „send an invite link to newlead@example.org“ or „how many MAUs did customer Foo have in June“), but while I almost got there, communicating an ever-growing set of actionable commands (with an arbitrary number of…

Care to share what made it complex? My comment above was most likely ignorant, but my general thought was to write some header prompt about available actions that the LLM could map to, and then ask it if a given input text matches to a pre-defined action. Much like what TypeChat does.

Does this sound similar enough to what you were doing? Was there something difficult in this that you could explain?

Aside from being completely hand-wavey in my hypothetical guess-timated implementation, i had figured the most difficult part would be piping complex actions together. "Remind me tomorrow about any events i have on my calendar" would be a conditional action based on lookups, etc - so order of operations would also have to be parsed somehow. I suspect a looping "thinking" mechanism would be necessary, and while i know that's not a novel idea i am unsure if i would nonetheless have to reinvent it in my own tech for the way i wanted to deploy.

Re: TypeChat

#74

Post by LordDragonfang »

BoorishBears wrote: post #71
bwestergard wrote: post #40

Earlier quoted context omitted.

The value is in: 1. Running the typescript type checker against what is returned by the LLM. 2. If there are type errors, combining those into a "repair prompt" that will (it is assumed) have a higher likelihood of eliciting an LLM output that type checks. 3. Gracefully handling the cases where the heuristic in #2 fails. https://github.com/microsoft/TypeChat/blob/main/src/typechat... In my experience experimenting wi…

Here's a project that does that better imo: https://github.com/dzhng/zod-gpt And by better I mean doesn't tie you to OpenAI for no good reason

I don't know where all you people work that your employer would prefer a random git repo (that has no support and no guarantee of updates) over a solution from Microsoft. (Alternatively: that you have so much free time that you'd prefer to fiddle with your own validation code instead of writing your actual app)

Open source solutions are great (which this still is, btw), but having a first-party solution is also a good thing.

Re: TypeChat

#75

Post by TechBro8615 »

verdverm wrote: post #27

I don't see the value add here. Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat... You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as mod…

Where's the vendor lock-in? This is an open source library and the file you linked to even includes configs for two vendors: ChatGPT and Bard.

Re: TypeChat

#76

Post by BoorishBears »

LordDragonfang wrote: post #74
BoorishBears wrote: post #71

Earlier quoted context omitted.

Here's a project that does that better imo: https://github.com/dzhng/zod-gpt And by better I mean doesn't tie you to OpenAI for no good reason

I don't know where all you people work that your employer would prefer a random git repo (that has no support and no guarantee of updates) over a solution from Microsoft . (Alternatively: that you have so much free time that you'd prefer to fiddle with your own validation code instead of writing your actual app) Open source solutions are great (which this still is, btw), but having a first-party solution is also a go…

I don't know which employer is hiring the people who make logical leaps like this but I thank them for their sacrifice.

At the end of the day the repo I linked is grokkable with about 10 minutes of effort, and has simple demonstrable usefulness by letting you swap out the LLM you're calling.

Both are experimental open source libraries in an experimental space.

Re: TypeChat

#77

Post by notRobot »

dvt wrote: post #72

This is my hot take: we're slowly entering the "tooling" phase of AI, where people realize there's no real value generation here, but people are so heavily invested in AI, that money is still being pumped into building stuff (and of course, it's one of the best way to guarantee your academic paper gets published). I mean, LangChain is kind of a joke and they raised $10M seed lol. DeFi/crypto went through this phase 2…

I feel like this is actually a very sensible take. AI has many uses, and it can be really good at some things, but it's not the hail mary it's being treated as.

Re: TypeChat

#78

Post by coffeemug »

dvt wrote: post #72

This is my hot take: we're slowly entering the "tooling" phase of AI, where people realize there's no real value generation here, but people are so heavily invested in AI, that money is still being pumped into building stuff (and of course, it's one of the best way to guarantee your academic paper gets published). I mean, LangChain is kind of a joke and they raised $10M seed lol. DeFi/crypto went through this phase 2…

When I use ChatGPT I feel like I'm looking at a different technology than other people. It's supposed to be able to answer every question and teach me anything, but in practice it turns out to be a content-farm-as-a-service (CFaaS?) Copilot is similar, it's usually easier for me to write the code than iterate through it to find the least bad example and then fix the bugs.

That said, AlphaGo went from "hallucinating" bad moves to the best player in the world in a fairly short period of time. If this is at all doable for language models, GPT-x may blow all this out of the water.

Re: TypeChat

#79

Post by obiefernandez »

If I can use this instead of functions, it's gonna save me a buttload of API usage, because the Typescript interface syntax is so concise. Can't wait to try it.

Re: TypeChat

#80

Post by akavi »

rezonant wrote: post #55
akavi wrote: post #20

Earlier quoted context omitted.

Typescript's type system is much more expressive than the one the function call feature makes available. I imagine closing the loop (using the TS compiler to restrict token output weights) is in the works, though it's probably not totally trivial. You'd need: * An incremental TS compiler that could report "valid" or "valid prefix" (ie, valid as long as the next token is not EOF) * The ability to backtrack the model I…

For the TS compiler: If you took each generation step, closed any partial JSON objects (ie close any open `{`), checked that it was valid JSON and then validated it using a deep version of Partial , that should do the trick.

Not for even the simplest schemas.

Eg, given even the type:

    {"aLongerKey": "value"}
The generation prefix:

    {"a
would by your algorithm produce the following invalid output:

    {"a}
Post reply on HN