Chatbots: Still dumb after all these years

Short of any real break-through in AI, I feel that a chatbot just isn't something most consumers want to use.

A few years ago I bought into the chatbot hype for a while (the Facebook Messenger API opening up for building chatbots was a major catalyst for that particular hype cycle; that was when I was into it), and explored a lot in building one and other AI services that helped build one. It was quickly apparent that for the most part it's simply an iteration of the "answer machine navigation tree" that 800 numbers already had, just in text form.

There was a widespread notion at the time that users wanted chatbots for reasons like wanting to speak in a human langauage with a company to get what they need and solve problems as well as having context from chat history. I think the industry has confused the mechanism (the chat) with the intention (speaking with a human). Consumers prefer a chat sure, that's because they don't want to waste time being on hold on a phone (anyone who has called customer service at any big org or the gov't would know); but ultimately their intention is to speak with a human that can solve problems. When problems arise that require speaking to a human, that's usually not something a bot or a program could solve.

This is polar opposite to any experience I've had with children. Children are aware of their environment and have complex thoughts, but sometimes they are unable to convey those thoughts with words. Children seem to remember conversations, and if I were to say "Go get me a red lego" and then subsequently say "now a green one" there is no ambiguity or confusion.

To me as these bots have "advanced" it has only highlighted how absurdly far we are from anything even approaching actual intelligence, even the intelligence of a toddler. The contextual awareness I have seen in bots is not much more than a cheap trick that is trivially fooled in scenarios that would not fool a child.

There will not, in the foreseeable future, be a model/ensemble/pipeline capable of conversing at the level of a human representative. Thinking of them as "humans, but cheaper" is destined to fail.

There are, however, use-cases where communicating with a huge group of individuals on a one-on-one level is useful. For example, years ago a company called Mainstay (then AdmitHub) built a chatbot for Georgia State designed to reduce "summer melt," a phenomenon in which intending freshmen drop off between graduating high school and enrolling in classes. For this use case, getting human-level conversation wasn't as critical as effective performance on tasks like question answering. Their bot, Pounce, was credited with a ~20% decrease in summer melt. They have a proper study here: https://mainstay.com/case-study/how-georgia-state-university...

This is an example, in my opinion, of the sweet-spot where intelligent, individualized, but not-near-human-level communication is needed, at a scale that goes beyond what you could reasonably do manually without an enormous dedicated staff.

Now, I don't know how numerous those opportunities are, or if they come anywhere close to the level of hype and funding chatbot companies have received. I just think that in a vacuum, there is a set of problems where they have value.

They were once advertised to help your granny use the website but I haven't met a chatbot that could help me do something.

How do you know it did? I.e. that a human it was passed to didn't just pass your inverted Turing test!