Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TypeChat

microsoft.github.io

Post reply on HN
Print view View original

91–100 of 174 posts

Re: TypeChat

#91

Post by jacamera »

katamaster818 wrote: post #36
tehsauce wrote: post #34

Earlier quoted context omitted.

maybe this function? https://github.com/microsoft/TypeChat/blob/4d34a5005c67bc494...

yup, just found that, super neat, I am 100% interested in using this for other runtime validation... It's interesting because I've always been under the impression the TS team was against the use of types at runtime (that's why projects like https://github.com/nonara/ts-patch exist), but now they're doing it themselves with this project... I wonder what the performance overhead of starting up an instance of tsc in me…

Great point. They're against it unless you're running it in a loop and paying for every API call!

Re: TypeChat

#92

Post by verdverm »

bwestergard wrote: post #50
verdverm wrote: post #42

Earlier quoted context omitted.

these are trivial steps you can add in any script, as your link demonstrates. Why would I want to add all this extra stuff just for that? The opaque retry until it returns valid JSON? That sounds like it will make for many pleasant support cases or issues Personally, I have found investing more effort in the actual prompt engineering improves success rates and reduces the need to retry with an appended error message.…

There are many subtleties to invoking the typescript type checker from node. It's nice to have support for that from the team that maintains the type checker.

Is the team working on typescript in a good position to be making LLM libraries, interfaces, and abstractions? Do they have the background and context to understand how their library fits into AI workflows? Could they have provided the same value with a blog post and sample code?

Re: TypeChat

#93

Post by TechBro8615 »

verdverm wrote: post #85
TechBro8615 wrote: post #75

Earlier quoted context omitted.

Where's the vendor lock-in? This is an open source library and the file you linked to even includes configs for two vendors: ChatGPT and Bard.

vendor lock in to a library and the design choices they make basically, since it reduces the user input space, you are giving up flexibility and control for some questionably valuable abstractions, such as a predefined prompt, no ability to prompt engineer, CoT/ToT, etc... if anything, choose a broader framework like langchain and have something like this an extension or plugin to the framework, no need for a library…

Weird, I would suggest the opposite - LangChain is a nuke that was hastily assembled to crack a peanut, almond, and whatever other nuts were hype driven into the framework. It's a mess of spaghetti - which is nothing against the Langchain authors - it was just the first iteration in a new problem space. But adopting it in a new codebase is a big commitment that locks you into complexity you'll almost certainly want to shed at some point.

Whereas this library is a much more focused approach that does one small thing well, and could be integrated into your own homerolled frameworks (or probably even langchain itself, assuming you use langchain.js).

Re: TypeChat

#94

Post by parentheses »

I'm very surprised that they're not using `guidance` [0] here.

It not only would allow them to suggest that required fields be completed (avoiding the need for validation [1]) and probably save them GPU time in the end.

There must be a reason and I'm dying to know what it is! :)

Side-note, I was in the process of building this very thing and good ol' Misrocoft just swung in and ate my lunch.. :/

[0] https://github.com/microsoft/guidance

[1] https://github.com/microsoft/TypeChat/blob/main/src/typechat...

Re: TypeChat

#95

Post by parentheses »

verdverm wrote: post #27

I don't see the value add here. Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat... You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as mod…

The value is turn unstructured data into structured data and ensure it satisfies schema constraints.

For example: you have 1000 free-text survey responses about your product, building a schema and for-each `TypeChat`ing them would get you a dataset for that free-text. It's mind-bogglingly useful.

Re: TypeChat

#96

Post by ofslidingfeet »

verdverm wrote: post #89
ofslidingfeet wrote: post #70

Earlier quoted context omitted.

Getting these models to reliably return a consistent structure without frequent human intervention and/or having to account for the personal moral opinions of big tech CEOs is not trivial, no.

There are multiple ways to get structured output, and what this library is doing is not really that interesting. The concept is interesting and has had multiple implementations already, the code (and abstraction) here is not interesting and creates more issues than it solves

Tell me how to get reliably structured output. I'm all ears.

Re: TypeChat

#97

Post by semiquaver »

BoorishBears wrote: post #71
bwestergard wrote: post #40

Earlier quoted context omitted.

The value is in: 1. Running the typescript type checker against what is returned by the LLM. 2. If there are type errors, combining those into a "repair prompt" that will (it is assumed) have a higher likelihood of eliciting an LLM output that type checks. 3. Gracefully handling the cases where the heuristic in #2 fails. https://github.com/microsoft/TypeChat/blob/main/src/typechat... In my experience experimenting wi…

Here's a project that does that better imo: https://github.com/dzhng/zod-gpt And by better I mean doesn't tie you to OpenAI for no good reason

How does TypeChat tie you to OpenAI more than zod-gpt does? The interface required of a chat completion model is as simple as it gets, and you can provide your own easily (as the linked post makes clear)

https://github.com/microsoft/TypeChat/blob/4d34a5005c67bc494...

Re: TypeChat

#98

Post by TillE »

I wish Copilot did something like this. I've found it'll regularly invent C# methods which don't exist, an error which seems trivial to catch and hide from the user. No output is better than bad output.

Re: TypeChat

#99

Post by gigel82 »

I agree with comments saying this is basically a 10-line "demo script" everyone could write and it is weird to have big names associated with it.

But I heard from MS friends that AI is an absolute "need to have". If you're not working on AI, you're not getting (as much) budget. I suspect this is more about ticking the box than producing some complex project. Unfortunately, throughout the company, folks are doing all kinds of weird things to tick the box like writing a "copilot" (with associated azure openai costs) fine-tuned on a handful of documentation articles :(

Post reply on HN