Natural language is an unnatural interface

> You ask it to show you a color picker, it generates HTML code for a color picker, you copy that into your browser and you can pick your color, which you can then copy&paste back into the LLM for further processing.

This is slower, more awkward, and less efficient than just picking a colour from an existing colour picker.

Not at all.

Programming languages are meant to be unambiguous. Natural languages are meant to be ambiguous.

They just solve different problems.

The best voice assistant I ever had was the one I DIY-ed around 2007, using Microsoft Speech API and a cheap piezoelectric mike I soldered to a long cable, hung off the wardrobe, and plugged into PC. The code itself was a mashup of MS SAPI demo of "controlled language" interface and some tutorials for how to control WinAMP with WM_USER messages in WinAPI. I designed a little tree of commands, maybe 3 level deep, wrote the magic XML for it, some trivial C++ logic for driving the voice recognizer and reacting to identified commands (including one that held the recognizer two levels deep in the command tree, so I could issue multiple commands from a subtree without having to repeat two extra words for each).

The result of this couple afternoons of working on this (instead of learning for my maturity exams, as I was supposed to), was a system that, I kid you not, was more reliable and delivered more value to me than any of the current voice assistants. For one, its recognition was flawless. The typical interaction would look like:

  $ Computer!
  > 
  $ Music, Playlist Alpha
  > 

I had commands for usual play/pause/resume, next/previous, four playlist (alpha through delta), and volume control at different granularity ("mute", "one quarter", "two quarters", "three quarters", "full", plus "louder" and "quieter" for IIRC +/- 5% or +/- 10% jumps). Plus some stubs for non-music thing that IIRC I never eventually implemented.

Here's the thing: it worked flawlessly. It heard me across the room. It heard me through music so loud that it was uncomfortable to talk in. It never self-triggered (except that one person who managed to make a swear word be read as the wake word, a single case out of many who tried). It worked fast - I could complete the whole command chain in less time than Google Assistant takes to start listening after "OK Google". The secret? Constrained grammar and training.

In order to use speech recognition in Windows back then, you had to turn it on and let it analyze a sample of your voice (offline! those were the days!), based on a recording of you reading some calibration text it gave you. This process was additive - you could repeat it to improve recognition accuracy. But a little known fact was that you could also supply your own text - and that was the other half that made the magic happen.

I created myself a training text, consisting of individual command words and their sequences, and trained the Windows speech recognition on it multiple times, under varying conditions. Specifically, I run:

{three locations in the room} x ({no background} + ({classical music, pop music, whatever was on FM radio} x {quiet playback, normal playback, very loud playback}))

training sessions. That's 30 sessions of repeating the same text. Each one took maybe a minute or less, so I was done with it in about an hour. And after that training, no matter where I was standing in the room and what I was doing, the voice control system worked with near-zero false positives and near-zero false negatives. I say "near" because I had maybe two or three cases of each, over months of continued use. And yes, I could play music so loud you couldn't talk in the room, and I could scream out commands, outshouting the music, and it would work. Try that with Google Assistant.

To recap: I had a system I hacked together in couple evenings, whose software was a relatively small tweak to a default example project (but done with love!) and hardware was hand-soldered from cheapest, locally-sourced parts, that did everything I wanted from a voice assistant, did it flawlessly, much faster than any of the voice assistants on the market today, completely off-line, in 2007, on a mid-range PC, without noticeably taxing its resources. This is why I occasionally rant that voice assistants are bloated and done backwards - all because they're designed to suit vendor needs first, user needs second.

--------

But hey, I know a way Google, Apple, Samsung (!) et al. could fix the shitty performance of their voice assistants and dictation software. They need to fine-tune a LLM on a dataset made of target words/sentences, and transcripts of them being misheard in great many ways. Then they need to feed the output of their voice-to-text pipeline through that LLM, so it can correct the text wholesale. That, or maybe, you know, do whatever Microsoft was doing in 2007 that made dictation work well and offline.

> Is it really that intimidating to have an empty text box on Whatsapp or your favorite SMS app? No, as you expect to have an appropriate response coming from the other side, pretty much regardless of what your input is.

Yes, very much yes - sure, I can expect to get an appropriate response, but that doesn't change the fact I don't know what to write about to start the conversation. Empty text box does indeed scare me - if I want to write something, but have no good idea what to write (or more than couple competing ideas that feel equivalent), my mind simply goes blank.

Yes, this applies to ChatGPT too. There's million of things I want to bounce off GPT-4. But when I have the time, none of those things come to mind.

Surely the author and I are not the only ones in this. There's a reason the "fear of empty page" is a term amongst writers. There's a reason you may occasionally hear of the "fear of empty text editor" in context of programming.

Of course, if I know what I want, then it's all fine - except, I find myself constantly constrained by my own typing speed. Doubly so now, with the recently improved response time of OpenAI's GPT endpoints.

Do you go to google.com when you have nothing to search for?

When you do, does the current opening UI feel that inadequate?

There's a simple reason they haven't added tons of UI elements for things like advanced search operators: the vast majority of the queries and the vast majority of users simply don't need them to get what they want from the tool.