Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chatbots: Still dumb after all these years

mindmatters.ai

Post reply on HN
Print view View original

211–220 of 426 posts

Re: Chatbots: Still dumb after all these years

#211

Post by shrdlu2 »

annexrichmond wrote: post #205
shrdlu2 wrote: post #200

Having worked on conversational AI (virtual agents/chatbots) for over a half a decade now, I can say that there are large differences in the capabilities of the solutions and the quality of implementations. Some are just bad and unhelpful. Some are very good. I'm personally familiar with several voice deployments doing millions of calls a month. Not only are there obvious cost savings but the calls handled (entirely)…

> the calls handled (entirely) by the virtual agents get better average cust sat metrics than human agents on the same set of calls does that take into account that the virtual agents are immediately available and human agents usually require a waiting room?

I'm sure that is some of it. I'd certainly be happier having my problem solved in 30 seconds rather than waiting in a queue for even 5 minutes.

Re: Chatbots: Still dumb after all these years

#212

Post by taylodl »

Chatbots are dumb - this is true, but a chat UI is smart. I work for a large utility in the U.S. We've recently resorted to text messaging for communicating with our customers, e.g. outage/restoral notifications, whether a payment is due, setting up payment arrangements if you've fallen behind on your payments, and so forth. Our customers are loving it, however they all are going in with the expectation this is a "dumb" chatbot. It's more like the interface of the early text adventure games - people know there's a very limited vocabulary and that ultimately the computer is dumb. Even so, as a UI it can be very useful. Plus there's no app to install and update, no web site to visit, you don't need to configure and manage notifications - it's simple. Even senior citizens with extremely limited computer skills are successful using it.

Re: Chatbots: Still dumb after all these years

#213

Post by tetha »

Teandw wrote: post #191
tetha wrote: post #93

Earlier quoted context omitted.

This is what our NLP teams are indeed working on. Sales prefers to describe it in more colorful ways, but practically, they are developing a more natural interface to an initial decision tree - you don't press 5 for a connectivity issue, you type "My fritzbox doesn't internet anymore" into a chatbox and it recognizes that as a connectivity issue. This goes on for 3-4 questions, until it either generates a case a huma…

Is the 5-10% savings on agents handling phone calls worth it when reputation damage is taken into consideration?

Most of our customers use these kinds of chat bots as an addition to their portfolio of support channels a user can use. If you don't like it, use another channel you prefer.

Re: Chatbots: Still dumb after all these years

#215

Post by serverlessmom »

Truthfully whenever I have been forced to interact with a chat bot designed by a business I have come to the conclusion that it is not there to help me, it is in fact there to frustrate me beyond belief and push me into giving up on seeking whatever help it is that I need. Chat bots while sometimes amusing often turn into a huge time sink as I try to interpret how it was programmed and how I can issue the correct commands to get me through to an actual person.

Re: Chatbots: Still dumb after all these years

#216

Post by AussieWog93 »

klabb3 wrote: post #195
version_five wrote: post #42

Earlier quoted context omitted.

I think a problem is the tighter cycle between academic discoveries and business people trying to monetize them. Large language models were developed, objectively a great achievement, and immediately someone nontechnical wants to apply their own interpretation and imagine that we can build a chatbot that you won't have to pay, and before you know it, people are selling and even deploying them. Anyone who questions or…

> Anyone who questions or points out that the technology doesn't do what the business people think it does[...] Uh oh, we've got a downer! Jokes aside, I'd like to consider an even simpler explanation, namely that "The purpose of a system is what it does"[1]. In this case, it would suggest decision makers are fully aware that they suck. Why would anyone want something that sucks? Because it's discouraging, and custom…

>Unsubscribing, replacements, returns, special requests are all costs, and that's a fixable problem in the current business doctrine, especially in the faceless megacorp flavor of business.

I'd believe that. Especially as processes mature and you really get your QA down pat, a good chunk of support requests tend to come from high-maintenance "black hole" customers who will only consume more of your resources when you help them.

Re: Chatbots: Still dumb after all these years

#217

Post by pdimitar »

pwinnski wrote: post #13

Having worked in ML at two different companies now, I think that people interpreting model output as intelligence or understanding says much more about the people than about the model output. We want it to be true, so we squint and connect dots and it's true. But it isn't. It's math and tricks, and if human intelligence is truly nothing more than math and tricks, then what we have today is a tiny, tiny, tiny fraction…

You didn't have to go for the jugular and do the brutal kill on the first hit, dude!

> We want it to be true, so we squint and connect dots and it's true.

That's exactly the issue. You summarized it once and for all and we can all go home and stop discussing it now and forever (until we get a general AI that is).

1. Normal people want to have intelligent machines. They watch movies and series and imagine one day a robot will cook for them so they actively look for intelligence, as you said. They buy Roombas and imagine they are going to clean 100% of their rooms (seen it, heard it, watched them rage when it didn't happen). They buy Alexa-enabled devices and imagine themselves like some aristocrats barking orders at an intelligent home (lol). But yeah, that's what the normal people do. It's quite puzzling to me.

2. People who work in the area are obviously biased and I have argued with some of them here on HN but I view it as a doomed affair. They insist there's a lot of innovation going on and that improvements are being done all the time yet we still have embarrassing failures as Michele Obama classified as a male and various politicians classified as known criminals or black people in general classified as gorillas. Like OK, where are your precious improvements and why are they NEVER finding their way into the news?

"It's hard to make someone understand something if their salary depends on them not understanding it", that's how I view the folks working in the DL / ML area. Sorry if that's offensive but just looking from the sidelines, it seems that what I say is true.

Re: Chatbots: Still dumb after all these years

#218

Post by jug »

scotty79 wrote: post #193

Just try to use GPT3 as a chatbot. It will blow your mind. Sure, it thinks it's 2017 and knows nothing about covid and if you ask it to explain math to you it will serve you pure garbage in most educational tone and manner, but it keeps context of the conversation orders of magnitude better than anything else. It doesn't feel like a bot. More like a dumb and confident human.

GPT3 chatbots are what the majority of the article is about.

Re: Chatbots: Still dumb after all these years

#219

Post by scotty79 »

jug wrote: post #218
scotty79 wrote: post #193

Just try to use GPT3 as a chatbot. It will blow your mind. Sure, it thinks it's 2017 and knows nothing about covid and if you ask it to explain math to you it will serve you pure garbage in most educational tone and manner, but it keeps context of the conversation orders of magnitude better than anything else. It doesn't feel like a bot. More like a dumb and confident human.

GPT3 chatbots are what the majority of the article is about.

Yeah, but it's different to read about it and converse with it yourself. It feels like a human, just the one that doesn't care about you or the truth.

There was a research about plugging such models into the fact database and it resulted in better answers with smaller model.

Re: Chatbots: Still dumb after all these years

#220

Post by harpersealtako »

Sharlin wrote: post #35
klyrs wrote: post #12

My favourite flaw of chatbots is exposed by ELIZA. Not chatting with ELIZA, (though, it does suffer this flaw) but using responses inspired by that program. "Please elaborate on that" or "tell me more about [noun]" etc. Bots appear to have zero lines of short term memory, and utterly fail to pick up a reference to the thing that they just said. My favorite being bot> [something plausibly human-sounding] me> What do y…

The GPT2 version of the AI Dungeon [1] could keep track of context for maybe couple of lines at a time. I've heard the GPT3 version is substantially better. The problem is, of course, that these "AI" chatbots on websites, marketing buzzwords notwithstanding, have very little to do with state-of-the-art machine learning, and are indeed unlikely to be any more sophisticated than the original ELIZA for the most part. [1…

AI Dungeon (and its successors like novelai [1] and holoAI [2] following its infamous censorship purges and mutinies) is basically designed entirely around that problem and has a lot of clever solutions that let writers keep relevant context in memory in a story that is longer than the model's maximum input length.

* "Memory", which is a customizable block of text that is repeated at the top of each API input. If your story revolves around your character being a vampire who melts in the sunlight, you can include something like "you are a vampire who will die if you go out in the sunlight" in the memory, and even 1000 characters back a paragraph of context can prime the AI accordingly.

* "Author's Note", which is a customizable short block of text that is inserted invisibly a few lines before your current place in the story when it's sent to the API. A note such as "[A/N: The following section includes depictions of graphic violence]" or "The following section is written in Shakespearean old English", as obvious and blunt as it might seem, actually works surprisingly well for nudging the AI towards a certain style or content.

* "World Info", which is a customizable dictionary of short text blocks that are conditionally added to the top of the API input like memory when a certain key appears in the current context. Imagine you have a story with 10 important characters who cycle in and out of the story. If you create an entry in the world info about Bob, then when you write that "Bob appears from behind the shrub", the blurb about Bob is automatically tacked on to the context so long as Bob is mentioned by name in the last few dozen inputs.

In general, both GPT-3 and the open source alternatives by EleutherAI such as GPT-J-6B are able to use a context primer from 1000 tokens prior to affect the current tail of a story. It's actually kind of uncanny how good they are at it -- you can have a story that in the memory at the top says "goblins always have purple skin" and notice that the AI will mention it as an offhand detail much farther down in the context.

[1] https://novelai.net/ [2] https://writeholo.com/

Post reply on HN