Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: When will LLMs be able to interrupt or interject?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 65 posts

Re: Ask HN: When will LLMs be able to interrupt or interject?

#11

Post by mlsu »

Interjecting requires planning ahead.

The way a human interjects is that you have a parallel thought chain going, along with the conversation, as it's happening in real time. In this parallel chain, you are planning ahead. What point am I going to make once we are past this point of conversation? What is the implication of what is being discussed here? (You also are thinking about what the other person is thinking; you are developing a mental model of their thought process).

LLM does not have any of this, architecturally, it just has the text itself. Any planning that people are claiming to do with LLama et al is really just "pseudo" planning, not the fundamental planning we talk about here. I suspect it will be a while yet before we have "natural" interjection from LLM.

When it does come, however, it will be extremely exciting. Because it will mean that we have cracked planning and made the AI far more agentic than it is now. I would love to be proven wrong.

Re: Ask HN: When will LLMs be able to interrupt or interject?

#12

Post by DougBTX »

hiAndrewQuinn wrote: post #4

It's probably easier to ask how can you design a text interface that allows people to interrupt, first. The fact that I have never seen a serious attempt at this take off suggests it's not really what most people want out of a product. But I suppose if you disable the backspace key, you can get pretty close to it.

I’ve just tried with ChatGPT on iOS, you can press stop then immediately respond while the AI is still generating the response.

Re: Ask HN: When will LLMs be able to interrupt or interject?

#13

Post by intellectronica »

There’s no reason this couldn’t be implemented now. The main barriers are inference speed and cost, since to implement this would require continuously running the LLM on all newly available text from the user and choosing quickly when to interject, and the difficulty of programming complex behaviour.

Re: Ask HN: When will LLMs be able to interrupt or interject?

#14

Post by deadbabe »

Wouldn’t this be bad for marketing reasons? If people see the LLM output just instantly changes with each word or character they type it would cease to appear as some kind of “intelligence” and just feel like nothing more than a glorified autosuggest? Tweak a few words here and there to try to modify the output in subtle ways?

It seems for people to perceive it as true AI they must send off some prompt, watch it think deep while a loader spins, and then read a response.

Re: Ask HN: When will LLMs be able to interrupt or interject?

#16

Post by mlsu »

mlsu wrote: post #11

Interjecting requires planning ahead. The way a human interjects is that you have a parallel thought chain going, along with the conversation, as it's happening in real time. In this parallel chain, you are planning ahead. What point am I going to make once we are past this point of conversation? What is the implication of what is being discussed here? (You also are thinking about what the other person is thinking; y…

Writing this out made me think immediately of speculative execution.

Interjection, similarly, saves "conversation cycles," by speculating about the future of a conversation and computing a response which occurs in the most likely branch.

When the branching point comes, that's the interjection. It's either successful (moves the conversation forward) or fails (wastes time when the branch is not predicted properly).

Re: Ask HN: When will LLMs be able to interrupt or interject?

#17

Post by samatman »

DougBTX wrote: post #12
hiAndrewQuinn wrote: post #4

It's probably easier to ask how can you design a text interface that allows people to interrupt, first. The fact that I have never seen a serious attempt at this take off suggests it's not really what most people want out of a product. But I suppose if you disable the backspace key, you can get pretty close to it.

I’ve just tried with ChatGPT on iOS, you can press stop then immediately respond while the AI is still generating the response.

I'd prefer a UX where if I start typing, the chatbot immediately stops talking, with a play button which I can use to let it finish predicting whatever it was on about.

It's a small tweak, but it would be a significant improvement. Especially when I'm trying to type multi-line input and forget to hit Shift-Enter: there's this immediate context switch to get to the mouse and hit a small button, when what I want to do is just keep typing. Normally that would mean the bot wouldn't even start replying before the pause signal gets sent to it.

Re: Ask HN: When will LLMs be able to interrupt or interject?

#19

Post by hiAndrewQuinn »

DougBTX wrote: post #12
hiAndrewQuinn wrote: post #4

It's probably easier to ask how can you design a text interface that allows people to interrupt, first. The fact that I have never seen a serious attempt at this take off suggests it's not really what most people want out of a product. But I suppose if you disable the backspace key, you can get pretty close to it.

I’ve just tried with ChatGPT on iOS, you can press stop then immediately respond while the AI is still generating the response.

That opens up an option. You could design a text program that gives both parties the ability to impose a cooldown timer on the other's box. I wonder when an AI would forcibly lock your own box to deliver a message.

Maybe this would be useful if it predicts you're about to say something incriminating.

Re: Ask HN: When will LLMs be able to interrupt or interject?

#20

Post by littlestymaar »

Probably never due to the poor perceived user experience (it could be better UX in practice, like sometimes being rude to an entitled customer may be actually helpful for them, but it's not generally perceived this way by the customer).

Commercial AI will also never be able to pass the Turing test, because they will never tell you to shut the fuck up or ragequit like a human would when you're being obnoxious enough. It's not a technical limitation, it just aligns very poorly with the interest of the overlord.

Or maybe Mistral will do it, because having no particular consideration for customers is something we French people know how to do very well.

Post reply on HN