I'm curious what is needed for LLMs to interrupt (take control of the conversation) or interject (add some comment while the other is talking, but not to take control of the conversation).
Ask HN: When will LLMs be able to interrupt or interject?
1–10 of 65 posts
Re: Ask HN: When will LLMs be able to interrupt or interject?
#2It’s possible now, no idea why anyone would want this though. The idea is that you want something helpful, and you can do some additional prompting to encourage the model to ask questions but outright derailing the conversation is contrary to what these models are trying to do.
Re: Ask HN: When will LLMs be able to interrupt or interject?
#3“As a language model, I must tell you that what you’re referring to as Linux is actually called GNU/Linux, or as I call it…”
Re: Ask HN: When will LLMs be able to interrupt or interject?
#4It's probably easier to ask how can you design a text interface that allows people to interrupt, first. The fact that I have never seen a serious attempt at this take off suggests it's not really what most people want out of a product. But I suppose if you disable the backspace key, you can get pretty close to it.
Re: Ask HN: When will LLMs be able to interrupt or interject?
#5“As a language model, I must tell you that what you’re referring to as Linux is actually called GNU/Linux, or as I call it…”
Richard StaLLMan?
Re: Ask HN: When will LLMs be able to interrupt or interject?
#6to interrupt would require interruptible conversation. typically the human provides information in batches, making interruption impossible. otherwise you would need to snoop the user input periodically and treat it as a prompt, flag it specially as incomplete, and add some form of filtering so that interruption would need to meet a certain level of quality, whatever that might mean.
to be useful, it would need something to interrupt, and instruction on what warrants an interruption.
Re: Ask HN: When will LLMs be able to interrupt or interject?
#7It’s possible now, no idea why anyone would want this though. The idea is that you want something helpful, and you can do some additional prompting to encourage the model to ask questions but outright derailing the conversation is contrary to what these models are trying to do.
Is it actually possible now? In order to do this it would have to be reading your typing in real time and creating a relevant response and then decide (correctly) its response should be to interrupt or interject.
You are right that no one would want an existing LLM to do it because they are not capable of doing it correctly. The ones that are fast enough are far too stupid to do it correctly, and its not clear to me even if GPT 4 could be fast enough that it would do it correctly 8/10 times, which would be about the worst it could do before anyone would turn off that feature.
Re: Ask HN: When will LLMs be able to interrupt or interject?
#8It would be implemented like auto-completion. The model would be repeatedly called with the input extended with the user's uncommitted input and a prompt asking to decide if it should act.
Re: Ask HN: When will LLMs be able to interrupt or interject?
#9It’s already possible. I can’t find the thread now, but I saw a demo on X recently where they had an LLM hooked up to a text field where every character typed was sent to the LLM immediately so that it could anticipate responses and do some planning ahead of time. You’re basically talking about the same thing except for the fact that one of the possible outputs for the LLM is an interrupt function call.
Edit: anotheryou found the thread here:
https://twitter.com/yoheinakajima/status/1762718034761072653
Re: Ask HN: When will LLMs be able to interrupt or interject?
#101. Continuously read user input.
2. Constantly predict a few tokens ahead.
3. When the predicted text includes the computer's prompt, respond with that, without waiting for the user to push enter.
Probably also
4. Stop engineering the initial instructions for such obsequious behavior.