Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chatbots: Still dumb after all these years

mindmatters.ai

Post reply on HN
Print view View original

91–100 of 426 posts

Re: Chatbots: Still dumb after all these years

#91

Post by deltarholamda »

tobyjsullivan wrote: post #43
arthur_sav wrote: post #5

I really hate when companies hide behind chatbots. Like, NOOOO, your "AI" chatbot didn't help me - get me an actual person.

Kind of a weird tangent but I had a very interesting experience with one of these customer support chatbots recently. My ISP apparently now requires that all customers communicate with their chatbot as the only point of contact. So, naturally, I was pseudo-outraged because I know chatbots are just a money-saving gimmick that reduces workload by driving away 80% of support requests regardless of whether they actually…

>That was the day I realized I have a deep-seated prejudice against chatbots that blinded me to the possibility that maybe, just maybe, they actually can help sometimes.

Nice try, Skynet, but we're on to you.

Re: Chatbots: Still dumb after all these years

#92

Post by pwinnski »

mike00632 wrote: post #47
pwinnski wrote: post #13

Having worked in ML at two different companies now, I think that people interpreting model output as intelligence or understanding says much more about the people than about the model output. We want it to be true, so we squint and connect dots and it's true. But it isn't. It's math and tricks, and if human intelligence is truly nothing more than math and tricks, then what we have today is a tiny, tiny, tiny fraction…

What if that's the case? What if me replying to you is nothing more than some atomically basic mechanism of statistical inference and randomness? Your entire perception of my personhood could be based on a sense of verisimilitude without significant reasoning to set me apart from a transcendant spirit. What then? Would you be less forgiving of my mistakes if you knew? Would you be less proud of work we collaborate on…

My point is the other way around. If I knew for certain that you and I were perfectly deterministic, it wouldn't change anything about how I viewed you (especially since some days I'm pretty sure we're all deterministically-driven), but it would suggest to me that our best efforts at AGI are probably at somewhere in the .00000000000001% range as complex as they need to be to even begin to approximate our human levels of intelligence or understanding. Or worse.

Re: Chatbots: Still dumb after all these years

#93

Post by tetha »

dexwiz wrote: post #45

Chatbots are about as useful as phone trees. They can help solve the top 5 easy/common problems, but they are useless passed that. Anyone who has worked in a call center knows that more than half the calls are about the same couple of issues: reset password, finding common help docs, etc. Since help desks are cost centers, it makes sense to half a robot handle as many of these as possible. I think most of the hate di…

This is what our NLP teams are indeed working on. Sales prefers to describe it in more colorful ways, but practically, they are developing a more natural interface to an initial decision tree - you don't press 5 for a connectivity issue, you type "My fritzbox doesn't internet anymore" into a chatbox and it recognizes that as a connectivity issue.

This goes on for 3-4 questions, until it either generates a case a human will look at, or until it can provide an FAQ entry fitting your problem. From a customers perspective, avoiding 5 - 10% support calls makes monetary sense, and from a (test-) user perspective, it's suprisingly decent if the knowledge base behind it is solid. And yes, "Fritzbox doesn't internet properly anymore" actually had helpful results.

Re: Chatbots: Still dumb after all these years

#95

Post by pwinnski »

kingkawn wrote: post #72
tomrod wrote: post #40

Earlier quoted context omitted.

> There is nothing about our reaction to the pre-conscious language these bots are displaying that doesn’t fall in line with our own normal development patterns Well... yes and no. Deployed models typically learn in a defined pattern, if at all. Various forms of data freshness, etc. to develop. But the chatbots don't have good history recall, typically, and know that what you mentioned 50 messages ago is relevant to…

You misunderstood what I was saying. I know the chatbot itself is not structured as we are. I’m saying that our reactions to them are the standard tools of mind-building that we apply to our own kids (and pets).

If I understand you, you're saying that we see patterns of intelligence or understanding in ML models in the same way we see patterns of intelligence or understanding in children or animals?

If so, I agree. I think that's our big flaw, in fact, because we instinctually apply patterns from birth, even when those patterns shouldn't be applied. So we see faces in the moon or on mars that aren't there. We see shapes moving in the dark that don't exist. And we seem to believe that ML models will develop over time as children or animals do, based on nothing more than our perceptions of similarity, our instinct to apply patterns even when we shouldn't.

Unlike a baby human, that ML model isn't going to develop increased complexity of thought over time. It's already maxed out. New models might up the complexity slightly, but that baby is going to vastly surpass any existing model in weeks or days.

Re: Chatbots: Still dumb after all these years

#96

Post by everdrive »

api wrote: post #8
ramesh31 wrote: post #4

Earlier quoted context omitted.

>Yet another Next Big Thing that I was right about predicting would fail. I was hired at my current company 4 years ago to work on our chatbot client. Despite years of engineering effort put into an NLP based decision tree system, over 90% of chats were still falling back to a live agent. They just recently sunsetted the product for lack of interest. Thank god that fad is over. Web design trends come and go, but that…

I got the impression the entire thing was pushed by Slack probably by way of some PR firm. I utterly despise how mindlessly faddish this field is. For a field supposedly so full of smart people who often pride themselves on being "contrarian" (sometimes to a fault), people sure do bleat like sheep when the next fad comes along.

This should not be read as a disagreement with anything you’ve said, but smart people are still human. We’re all susceptible to the same flaws, but those flaws are just expressed with more complexity as intelligence increases.

Re: Chatbots: Still dumb after all these years

#97

Post by perryizgr8 »

parkingrift wrote: post #54
kingkawn wrote: post #32

Earlier quoted context omitted.

That interpretation of communication is how we develop and craft the personalities of children. There is nothing about our reaction to the pre-conscious language these bots are displaying that doesn’t fall in line with our own normal development patterns. And in the long run the same desire will lead us to develop bots that are capable of thinking.

These bots seem equivalent to an adult with amnesia after every spoken sentence. Absolute understanding of the language, and some impressive display of recalling facts, but without any understanding of the environment or context of the conversation. This is polar opposite to any experience I've had with children. Children are aware of their environment and have complex thoughts, but sometimes they are unable to conve…

When you talk to children who haven't developed the skill of talking completely, you still get the sense that there's a lot going on inside that they're unable to express. Sometimes they will show it to you with their actions. I wonder if chat bots are also struggling with the language, but have an internal story that's playing out, desperate to be understood. But they can't, because the only interface they have is a stream of text.

Re: Chatbots: Still dumb after all these years

#99

Post by KittenInABox »

melony wrote: post #48
version_five wrote: post #42

Earlier quoted context omitted.

I think a problem is the tighter cycle between academic discoveries and business people trying to monetize them. Large language models were developed, objectively a great achievement, and immediately someone nontechnical wants to apply their own interpretation and imagine that we can build a chatbot that you won't have to pay, and before you know it, people are selling and even deploying them. Anyone who questions or…

That's a good thing. The market is very good at empirically validating research. It helps filter out the chaff and ensure research is useful. Better than academia wasting years on concepts like the "semantic web" where nobody can even agree on what it means. Academia is under attack from many political directions right now, being able to show useful output will help it thrive in the long run.

> The market is very good at empirically validating research. It helps filter out the chaff and ensure research is useful.

I am very skeptical and would like to see studies that evidence this. Counterfactually, the market would never tolerate the decades of research behind MRNA vaccines and the researchers behind it were considered useless. The market has also put in mind-blowing amounts of money towards the ameloid hypothesis behind alzheimers to massive disappointment.

Re: Chatbots: Still dumb after all these years

#100

Post by dkarl »

version_five wrote: post #42
pwinnski wrote: post #13

Having worked in ML at two different companies now, I think that people interpreting model output as intelligence or understanding says much more about the people than about the model output. We want it to be true, so we squint and connect dots and it's true. But it isn't. It's math and tricks, and if human intelligence is truly nothing more than math and tricks, then what we have today is a tiny, tiny, tiny fraction…

I think a problem is the tighter cycle between academic discoveries and business people trying to monetize them. Large language models were developed, objectively a great achievement, and immediately someone nontechnical wants to apply their own interpretation and imagine that we can build a chatbot that you won't have to pay, and before you know it, people are selling and even deploying them. Anyone who questions or…

There isn't really a new "problem" with AI. Businesses love hype, they love anything that can punch up a sales pitch, they love shitty cheap automation that replaces expensive human work, they love to identify ways to lower quality and save money without affecting revenue, and AI can be good for all of those things without being good at anything.
Post reply on HN