Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TypeChat

microsoft.github.io

Post reply on HN
Print view View original

131–140 of 174 posts

Re: TypeChat

#131

Post by jlnho »

Edes wrote: post #59
jlnho wrote: post #43

Earlier quoted context omitted.

It's because not everyone can be as gifted as you. I think the (arguably very prototypical) implementation is not what's interesting here. It's the concept itself. Natural language may soon become the default interface for most of the computing people do on a day to day basis, and tools like these will make it easier to create new applications in this space.

I'm gonna love trying to figure out what query gets the support chatbot to pair me with an actual human so that I can solve something that's off script

Ideally you would jutst click the "talk to a human" button, but what do I know?

Re: TypeChat

#132

Post by abhinavkulkarni »

There already are techniques to guade LLMs into producing output that adhere to a schema. For e.g. forcing LLMs to stick to a Context-Free Grammar: https://matt-rickard.com/context-free-grammar-parsing-with-l...

Just like many similar methods, this is based on logit biasing, so it may have an impact on quality.

Re: TypeChat

#133

Post by quickthrower2 »

verdverm wrote: post #27

I don't see the value add here. Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat... You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as mod…

You can probably define the python language grammar as a typescript type though!

Re: TypeChat

#134

Post by throwaway290 »

bottlepalm wrote: post #116
dbish wrote: post #57

Earlier quoted context omitted.

OpenAI is pretty likely working on their own (see Kaparthy's "Building a kind of JARVIS @ OреոΑӏ"), and Microsoft of course is doing an integration or reinterpretation of Cortana with OpenAI's LLMS (since they are incapable of building their own models nowadays it seems - "Why do we have Microsoft Research at all?”-S.N.), but there's a lot less value in voice driven LLM then there is in actually being able to perform…

It's July, they just needed to put a voice interface on ChatGPT, it'd easily help them sell more pro licenses as well. I'm not a conspiracy person, but this just seems so obvious it feels like there's something else going on here.

No big company wants their appliance to accidentally talk customer's child into suicide or spouse into a divorce. Bad for image.

Re: TypeChat

#135

Post by rob74 »

siva7 wrote: post #41
mahalex wrote: post #39

So, it's a thing that appends "please format your response as the following JSON" to the prompt", then validates the actual response against the schema, all in a "while (true)" loop (literally) until it succeeds. This unbelievable achievement is a work of seven people (authors of the blog post). Honestly, this is getting beyond embarrassing. How is this the world we live in?

One of the authors is Anders Hejlsberg, the guy behind c# and delphi

I think he's probably more of an author in the way that the leader of a research team is always credited on any paper by the team, even if he didn't personally do any actual work on it?

Anyway, TIL that Hejlsberg is also involved with TypeScript...

Re: TypeChat

#136

Post by rolisz »

andy_xor_andrew wrote: post #121

Here's one thing I don't get. Why all the rigamarole of hoping you get a valid response, adding last-mile validators to detect invalid responses, trying to beg the model to pretty please give me the syntax I'm asking for... ...when you can guarantee a valid JSON syntax by only sampling tokens that are valid? Instead of greedily picking the highest-scoring token every time, you select the highest-scoring token that co…

> when you can guarantee a valid JSON syntax by only sampling tokens that are valid? Instead of greedily picking the highest-scoring token every time, you select the highest-scoring token that conforms to the requested format.

Yes, you can guarantee a syntactically correct JSON that way, but will it be a semantically correct? If the model really really really wanted to put another token there, but you are forcing it to put a {, maybe the following generated text won't be as good.

I'm not sure, I'm just wondering out loud.

Re: TypeChat

#137

Post by mahalex »

lsh123 wrote: post #58
mahalex wrote: post #39

So, it's a thing that appends "please format your response as the following JSON" to the prompt", then validates the actual response against the schema, all in a "while (true)" loop (literally) until it succeeds. This unbelievable achievement is a work of seven people (authors of the blog post). Honestly, this is getting beyond embarrassing. How is this the world we live in?

Hm... so how do we know that the actual values in the produced json are correct???

As with anything output by “AI”: you don’t.

Re: TypeChat

#138

Post by geysersam »

rolisz wrote: post #136
andy_xor_andrew wrote: post #121

Here's one thing I don't get. Why all the rigamarole of hoping you get a valid response, adding last-mile validators to detect invalid responses, trying to beg the model to pretty please give me the syntax I'm asking for... ...when you can guarantee a valid JSON syntax by only sampling tokens that are valid? Instead of greedily picking the highest-scoring token every time, you select the highest-scoring token that co…

> when you can guarantee a valid JSON syntax by only sampling tokens that are valid? Instead of greedily picking the highest-scoring token every time, you select the highest-scoring token that conforms to the requested format. Yes, you can guarantee a syntactically correct JSON that way, but will it be a semantically correct? If the model really really really wanted to put another token there, but you are forcing it…

Well, if the output doesn't conform to the format it's useless. If the model can't produce good and correct output then it's simply not up to the task.

Re: TypeChat

#139

Post by mavamaarten »

9dev wrote: post #38
bottlepalm wrote: post #22

How does no voice assistant (Apple, Google, Amazon, Microsoft) integrate LLMs into their service yet, and how has OpenAI not released their own voice assistant? Also like RSS, if there were some standard URL a websites exposed for AI interaction, using this TypeChat to expose the interfaces, we'd be well on our way here.

Seriously, it feels like there’s some collusion going on behind the scenes. This is the most obvious use case for the technology, but none of the big vendors have explored it.

I think it's because it turns out that taming a generative language model is really difficult. It's what we need to support more than some hardcoded simple questions, but companies like Google who are known for search want to keep their image of "use us to find what you're looking for". In the current state, their models (especially Bard in my experience) simply return bullshit and want to sound confident. They need to get beyond that stage.

But I feel you. My Google Assistant doesn't even seem to look for answers to questions anymore. All I get, even for simple queries, is a "sorry, I don't understand".

Re: TypeChat

#140

Post by hnlmorg »

JSavageOne wrote: post #83
dvt wrote: post #72

This is my hot take: we're slowly entering the "tooling" phase of AI, where people realize there's no real value generation here, but people are so heavily invested in AI, that money is still being pumped into building stuff (and of course, it's one of the best way to guarantee your academic paper gets published). I mean, LangChain is kind of a joke and they raised $10M seed lol. DeFi/crypto went through this phase 2…

Hard disagree on AI being just a bubble with limited applicability. > It won't be able to self-drive cars due to all the edge cases, it won't be able to perform surgeries because it might kill people, etc. You literally just cherry-picked the most difficult applications of AI. The vast majority of peoples' jobs don't involve life or death, and thus are ripe for automation. And even if the life or death jobs retain a…

> By 2030 humans probably won't be typing code anymore, it'll just be prompting machines and directing AI agents. By then most peoples' jobs will also be automated.

Eventually, perhaps. But by 2023? Definitely not.

I think both you and the GP are at opposite ends of the extreme and the reality is somewhere in that gulf in between

Post reply on HN