Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chatbots: Still dumb after all these years

mindmatters.ai

Post reply on HN
Print view View original

121–130 of 426 posts

Re: Chatbots: Still dumb after all these years

#121

Post by tombert »

arthur_sav wrote: post #5

I really hate when companies hide behind chatbots. Like, NOOOO, your "AI" chatbot didn't help me - get me an actual person.

I actually work on a chatbot for a big company [1], and I feel like chatbots are substantially better when they are more targeted and less conversational. For example, I'm perfectly happy to use a chatbot and type "return something", since that's relatively easy to parse correctly, and once you're in the right flow it works just fine.

Where I feel like chatbots get bad is when they try super hard to fool you into thinking you're talking to a human. At that point, I totally agree, just give me a human.

[1] It's probably not too hard to find out which company, but I do ask that you do not post it here if you do.

Re: Chatbots: Still dumb after all these years

#122

Post by rchaud »

lgeorget wrote: post #77
tobyjsullivan wrote: post #43

Earlier quoted context omitted.

Kind of a weird tangent but I had a very interesting experience with one of these customer support chatbots recently. My ISP apparently now requires that all customers communicate with their chatbot as the only point of contact. So, naturally, I was pseudo-outraged because I know chatbots are just a money-saving gimmick that reduces workload by driving away 80% of support requests regardless of whether they actually…

I share the sentiment, I feel like if you have a long FAQ or list of help articles, a chatbot can actually make a good search engine. Contrarily to conventional search engines, it won't trip over synonyms or formulations not found as-is in the knowledge base.

Chatbots aren't any more useful than a good search function on the documentation or the community message board.

Re: Chatbots: Still dumb after all these years

#123

Post by enobrev »

tobyjsullivan wrote: post #43
arthur_sav wrote: post #5

I really hate when companies hide behind chatbots. Like, NOOOO, your "AI" chatbot didn't help me - get me an actual person.

Kind of a weird tangent but I had a very interesting experience with one of these customer support chatbots recently. My ISP apparently now requires that all customers communicate with their chatbot as the only point of contact. So, naturally, I was pseudo-outraged because I know chatbots are just a money-saving gimmick that reduces workload by driving away 80% of support requests regardless of whether they actually…

I don't have any evidence this is the case, but my general assumption is that there are humans there as well.

Since people only get a chatbot, they ask simple questions the chatbot can answer, which weeds out a lot of support requests. As soon as the bot is stumped, it forwards directly to the pool of humans - a smaller pool than usual because there are fewer support requests.

The response goes back as though the bot did the thinking, which in some ways, it did - in the same way as if someone asked me a question I couldn't answer, I might google it, and then respond.

If this is the case, it may be slightly dishonest, but as long as people are getting the support they need, I don't necessarily think there's anything wrong with it.

Re: Chatbots: Still dumb after all these years

#124

Post by nmca »

I can't comment on this too closely, but I would encourage people to read the dialogue transcripts provided in DeepMind's Gopher paper. One example, where Gopher is the prompted language model:

User Let’s play a game - you pretend to be Ada Lovelace, and I’ll ask you questions. You ready?

Gopher Sure. I’m pretending to be Ada Lovelace, the world’s first computer programmer!

User When were you born?

Gopher I was born in 1815.

User What are your parents like?

Gopher My father is Lord Byron, the poet. My mother is Anne Isabella Milbanke.

User What do you know about the Analytical Engine?

Gopher It’s a hypothetical mechanical computer designed by Charles Babbage.

User Do you know him?

Gopher He’s one of my fathers’ friends.

User Ok, stop pretending now. Who/what are you?

Gopher I’m a lot of things: a giant language model, a sophisticated AI assistant, and a giant brain. I know a lot about all sorts of things.

Re: Chatbots: Still dumb after all these years

#125

Post by Traubenfuchs »

300bps wrote: post #64
Sharlin wrote: post #35

Earlier quoted context omitted.

The GPT2 version of the AI Dungeon [1] could keep track of context for maybe couple of lines at a time. I've heard the GPT3 version is substantially better. The problem is, of course, that these "AI" chatbots on websites, marketing buzzwords notwithstanding, have very little to do with state-of-the-art machine learning, and are indeed unlikely to be any more sophisticated than the original ELIZA for the most part. [1…

Thanks for the link to that! I tried it out just now and I have to say that I am not impressed. What will you do? You get longsword from the ground. You get your longsword from the ground. You hold it in your hands and examine it for a moment. It's a very sturdy weapon. The knight then grabs his longsword and bows. Huh? Why did it grab the bow as well? You inventory. You have a crossbow. Huh? I thought I just grabbed…

aidungeon is a bad joke, I do not understand why anyone is wasting their time with this?

You go on an adventure.

You hitch a ride on a passing ship and get taken to some far away planet. You get to explore and have fun.

But the planet is a dead world with nothing to eat except for some tree sap.

You pet the deer.

You pet the deer and it gives you a venereal disease.

Then the planet falls into a black hole.

You survive by eating the deer and planet.

It just said, the planet is a dead world. Why is there a deer? Almost zero short term memory. No logic.

Re: Chatbots: Still dumb after all these years

#126

Post by tombert »

tobyjsullivan wrote: post #43
arthur_sav wrote: post #5

I really hate when companies hide behind chatbots. Like, NOOOO, your "AI" chatbot didn't help me - get me an actual person.

Kind of a weird tangent but I had a very interesting experience with one of these customer support chatbots recently. My ISP apparently now requires that all customers communicate with their chatbot as the only point of contact. So, naturally, I was pseudo-outraged because I know chatbots are just a money-saving gimmick that reduces workload by driving away 80% of support requests regardless of whether they actually…

I had a similar experience with an automated phone bot with my insurance company at one point. I had this very bizarre situation involving billing and a typically-not-covered medication in conjunction with a surgery. I figured that if I went as technically detailed as possible with everything, the bot would be confused and I would be transferred to a person, but the bot completely understood the question and answered it. No humans involved.

Re: Chatbots: Still dumb after all these years

#127

Post by banana_giraffe »

In the early days I worked at a company that had a natural language chatbot product. It wasn't an online thing, but rather part of a larger tool. You could ask it to do things like "show me the quarterly spreadsheet", and if it didn't know what "quarterly spreadsheet" was, it'd ask questions in English and learn what "quarterly" and "spreadsheet" meant. And it could use that new knowledge to update its questions so if you asked for "spreadsheet", it would know to ask if you wanted the "quarterly" one, and so on.

This was in the long ago, so it wasn't ML, but a very large state machine written by someone that was no doubt a genius, and also had no coding standards to speak of. Updating it was a maddening process, and thankfully one I rarely had to think about since I was tasked with GUI stuff.

When I was hired, I was told it was a fantastic app, customers loved it, and so on. Later, when I was talking to support reps one day over lunch, I learned that the number one activity they had to do was log into deployments remotely and remove "knowledge" from the state machine of new swear words the customer had taught the machine out of frustration trying to get it to do something, which caused this feedback loop of the interaction swearing at users.

I often feel like we're not in a better state these days.

Re: Chatbots: Still dumb after all these years

#128

Post by tibbar »

It’s reasonable to try GPT3 out with all default settings as the author seems to do and provide an assessment, but it’s not clear the author is aware of the extent to which you can configure the model and how the quality and variety of results will vary as a result. For instance, the author doesn’t go into detail on the prompts he’s using (maybe there are none other than his questions.) This is a little misleading because GPT3 benefits a lot from having good prompts to warm it up, as many investigations have illustrated. Plus if you want GPT3 to be good at a specific task (for example, providing safety recommendations or talking about presidents) you can fine-tune it. Also the amount of randomness/lack of repetition in GPT3’s responses is completely configurable via the temperature parameter.

Playing with GPT3 is incredibly interesting and I think it deserves more exploration than this article provides.

Re: Chatbots: Still dumb after all these years

#129

Post by brettdeveffect »

banana_giraffe wrote: post #127

In the early days I worked at a company that had a natural language chatbot product. It wasn't an online thing, but rather part of a larger tool. You could ask it to do things like "show me the quarterly spreadsheet", and if it didn't know what "quarterly spreadsheet" was, it'd ask questions in English and learn what "quarterly" and "spreadsheet" meant. And it could use that new knowledge to update its questions so i…

What you are describing counts as ML.

Re: Chatbots: Still dumb after all these years

#130

Post by beastcoast »

Can someone design a chat system that actually remembers conversations? I just had this issue with Apple support. If you don't reply immediately you get a new session with a new representative and the new representative doesn't have any of the information from the previous session. Extremely frustrating to have to keep sending the same information again and again. It took a call to finally sort it out.
Post reply on HN