Chat is a bad interface for tools in general, but this problem goes deeper than that.
What's a good interface?
There are a few things we try to balance to make a good UI/UX:
- Latency: How long it takes to do a single task
- Decision-tree pathing: How many tasks to meet a goal
- Flexibility/Configurability: How much of a task can be encapsulated by the user's predefined knowledge of the system
- Discoverability: What tasks are available, and where
The perfect NLP chat could accomplish some of these:
- Flexibility/Configurability: Define/infer words and phrases that the user can use as shortcuts
- Decision-tree pathing: Define concepts that shortcut an otherwise verbose interaction
- Latency: Context-aware text-completions so the user doesn't need to type as much
- Discoverability: Well-formed introductions and clarifying questions to introduce useful interaction
This can only get us so far. What better latency can be accomplished than a button or a keyboard shortcut? What better discoverability than a menu?
The most exciting prospect left is flexibility. Traditional software is inflexible. It can only perform the interaction it was already designed with. Every design decision becomes a wall of assumption. These walls are the fundamental architecture of software. Without them, we would have nothing. With them, we have a structure that guides us along whatever assumptions were already made.
If we want to change something about our software's UI, then we must change the software itself, and that means writing. If NLP was a truly solved problem, then software compatibility and flexibility would be trivialized. We could redesign the entire UI by simply describing the changes we want.
LLMs are not even close. Sure, you can get one to generate some code, but only if the code you want generated is close enough to the text it was already trained on. LLMs construct continuations of tokens: no more, no less. There is no logic. There is no consideration about what is right or wrong: only what is likely to come next.
Like you said,
> You can’t build real software without being precise about what you want.
This is the ultimate limitation of UI. If only we could be ambiguous instead! LLMs let us do that, but they keep that ambiguity permanent. There is no real way to tie an LLM back down to reality. No logic. No axioms. No rules. So we must either be precise or ambiguous. The latter option is an exciting development, and certainly offers its own unique advantages, but it isn't a complete solution.
---
I've been thinking through another approach to the ambiguity problem that I think could really give us the expressive power of natural language, while preserving the logical structure we use to write software (and more). It wouldn't solve the problem entirely, but it could potentially move it out of the way.