Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TypeChat

microsoft.github.io

Post reply on HN
Print view View original

151–160 of 174 posts

Re: TypeChat

#151

Post by dwaltrip »

dvt wrote: post #72

This is my hot take: we're slowly entering the "tooling" phase of AI, where people realize there's no real value generation here, but people are so heavily invested in AI, that money is still being pumped into building stuff (and of course, it's one of the best way to guarantee your academic paper gets published). I mean, LangChain is kind of a joke and they raised $10M seed lol. DeFi/crypto went through this phase 2…

> we're slowly entering the "tooling" phase of AI, where people realize there's no real value generation here

Hard disagree. A very clear counterexample from my usage:

Gpt-4 is phenomenal at helping a skilled person work on tangential tasks where their skills generally translate but they don’t have strong domain knowledge.

I’ve been writing code for a decade, and recently I’ve been learning some ML for the first time. I’m using gpt-4 everyday and it’s been a delight.

To be fair, I can see one might find the rough edges annoying on occasion. For me, it’s quite manageable and not much of a bother. I’ve gotten better at ignoring or working around them. There is definitely an art to using these tools.

I expect the value provided to continue growing. We haven’t plucked all of the low-hanging or mid-hanging fruit yet.

I can share chat transcripts if you are interested.

Re: TypeChat

#152

Post by bwestergard »

BoorishBears wrote: post #71
bwestergard wrote: post #40

Earlier quoted context omitted.

The value is in: 1. Running the typescript type checker against what is returned by the LLM. 2. If there are type errors, combining those into a "repair prompt" that will (it is assumed) have a higher likelihood of eliciting an LLM output that type checks. 3. Gracefully handling the cases where the heuristic in #2 fails. https://github.com/microsoft/TypeChat/blob/main/src/typechat... In my experience experimenting wi…

Here's a project that does that better imo: https://github.com/dzhng/zod-gpt And by better I mean doesn't tie you to OpenAI for no good reason

I use Zod a great deal day to day, so this is appealing inasmuch as it would allow me to re-use those definitions.

Re: TypeChat

#153

Post by verdverm »

sambroner wrote: post #120
msp26 wrote: post #101

Earlier quoted context omitted.

You're overrating the influence of the name Microsoft here. It's just some devs from the company working on this with no proper guarantee backing the project. I've been through this whole song and dance already with Microsoft's Guidance (another LLM project) and could not justify using it further in production at work. We built some tools and wrappers ourselves and it wasn't even that difficult. These libraries are o…

I’m pretty sure Anders, Steve Lucco, and Daniel Rosenwasser worked on this. So inventors + current lead PM of typescript. Should lend some credibility to the project.

Not really, better to leave the AI stuff to the AI people rather than PL people. When you don't, you get gimmick libraries like this rather than a solution that fits into the ecosystem

These folks have no pedigree when it comes to LLMs or AI, so no it does not lend credibility

Re: TypeChat

#154

Post by verdverm »

icholy wrote: post #109
verdverm wrote: post #92

Earlier quoted context omitted.

Is the team working on typescript in a good position to be making LLM libraries, interfaces, and abstractions? Do they have the background and context to understand how their library fits into AI workflows? Could they have provided the same value with a blog post and sample code?

Your coworkers must love you.

Indeed, we all do what we are good at and appreciate each other and no having to do the things they do

But what does your comment have to do with any of this at all?

Re: TypeChat

#155

Post by verdverm »

TechBro8615 wrote: post #93
verdverm wrote: post #85

Earlier quoted context omitted.

vendor lock in to a library and the design choices they make basically, since it reduces the user input space, you are giving up flexibility and control for some questionably valuable abstractions, such as a predefined prompt, no ability to prompt engineer, CoT/ToT, etc... if anything, choose a broader framework like langchain and have something like this an extension or plugin to the framework, no need for a library…

Weird, I would suggest the opposite - LangChain is a nuke that was hastily assembled to crack a peanut, almond, and whatever other nuts were hype driven into the framework. It's a mess of spaghetti - which is nothing against the Langchain authors - it was just the first iteration in a new problem space. But adopting it in a new codebase is a big commitment that locks you into complexity you'll almost certainly want t…

I agree that LangChain has some pretty poor APIs and abstractions, and I do even question the usefulness of what they provide.

But this library amounts to a loop around a very basic prompt and running the ts toolchain to produce an error message that is then appended to the prompt next iteration. It is not easily integrated into anything and is written by people who do not practice or develop AI.

Re: TypeChat

#156

Post by verdverm »

jameshart wrote: post #122
verdverm wrote: post #87

Earlier quoted context omitted.

the debate is about how valuable the abstraction here is to warrant a library, and the fact that it predefines the prompt and api call flow, so you cannot prompt engineer or use something like CoT/ToT

This amounts to saying ‘how dare someone publish some code that they wrote!’ Is it your impression that this is being pitched as some grand solution? That this was published as a way to shut out other people from doing the same thing in other ways? Can’t we just look at a cool thing someone did, and released for other people to play with, and say ‘huh! That’s neat!’ And get inspired?

except it's not neat or novel, this idea has been around and implemented for many months now, by many people, using many methods. Running a tool on the output and then feeding that back to the LLM, also not novel and a widely used technique

> We'd love to know if TypeChat is something that's useful and interests you!

We are providing feedback to them here

Re: TypeChat

#157

Post by _andrei_ »

Just use function calling, declare your function schema using Zod, and convert it to JSONSchema automatically. You don't have to write your types more than once, you get proper validation with great error messages, and can extend it.

Re: TypeChat

#158

Post by smrq »

bwestergard wrote: post #50
verdverm wrote: post #42

Earlier quoted context omitted.

these are trivial steps you can add in any script, as your link demonstrates. Why would I want to add all this extra stuff just for that? The opaque retry until it returns valid JSON? That sounds like it will make for many pleasant support cases or issues Personally, I have found investing more effort in the actual prompt engineering improves success rates and reduces the need to retry with an appended error message.…

There are many subtleties to invoking the typescript type checker from node. It's nice to have support for that from the team that maintains the type checker.

Admittedly, couldn't they spend some effort on making that invocation less subtle instead?

Re: TypeChat

#159

Post by waffletower »

LordDragonfang wrote: post #74
BoorishBears wrote: post #71

Earlier quoted context omitted.

Here's a project that does that better imo: https://github.com/dzhng/zod-gpt And by better I mean doesn't tie you to OpenAI for no good reason

I don't know where all you people work that your employer would prefer a random git repo (that has no support and no guarantee of updates) over a solution from Microsoft . (Alternatively: that you have so much free time that you'd prefer to fiddle with your own validation code instead of writing your actual app) Open source solutions are great (which this still is, btw), but having a first-party solution is also a go…

Many companies expressly avoid Microsoft products, particularly given its well exposed history of embrace, extend, extinguish.

Re: TypeChat

#160

Post by waffletower »

Reliance on strong typing for LLM output coercion is a potentially lossy and inefficient approach that can introduce redundant LLM queries and costs. LLM output is far more subtle than this. But the strongly typed hammer is very attractive to many developers, particularly those in the Typescript ecosystem.
Post reply on HN