Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chatbots: Still dumb after all these years

mindmatters.ai

Post reply on HN
Print view View original

261–270 of 426 posts

Re: Chatbots: Still dumb after all these years

#261

Post by systemvoltage »

simplestats wrote: post #206
sam0x17 wrote: post #143

Earlier quoted context omitted.

Capital is far too short-sighted to look beyond this year or this quarter. Many of the important groundbreaking ideas or projects take years, even decades to formulate. As I've said elsewhere, capital will just take the easiest road it is allowed to take, which currently is stifling innovation, buying competitors, maintaining monopolies, relying on economies of scale to shut out competitors when they can't be acquire…

I cant think of the last time I saw the word "corporations" used in a post that wasn't part of an attack on them as inferior/evil/villanous etc. Speaking of NLP, your post would be very easy to classify sentiment of using a simple keyword approach. Anyway, I've worked both in academia and industry. Industry is simply more practical and better at technology. In tech areas, academia desperately needs industry to provid…

Hot take: Corporations are also amazing. James Webb mirrors? Made by a private big corp (Ball Aerospace). So are all instruments of James Webb. NASA contracts out their work to many big fat corporations. Semiconductors to hearing aids, Corporations make the world tick. They’re absolutely incredible. When people get together, they can build stuff like rockets and vaccines.

Capitalism makes all of this happen. So the attack on capitalism is so deeply deluded, it’s strange to see it in a tech crowd who know better. Hell, your salary probably comes from “Corporations”. When you’re in the hospital, note all the corporations doing evil things to save your life.

Re: Chatbots: Still dumb after all these years

#262

Post by visarga »

TedShiller wrote: post #226

It's called artificial intelligence for a reason. Because it's not intelligence.

> It's called artificial intelligence for a reason. Because it's not human intelligence.

FTFY

It's slower at learning basics, slower at learning new tasks, makes silly mistakes often, but it's growing at a speed much faster than biological evolution. And when there is extensive data it surpasses humans.

Re: Chatbots: Still dumb after all these years

#263

Post by crate_barre »

pwinnski wrote: post #13

Having worked in ML at two different companies now, I think that people interpreting model output as intelligence or understanding says much more about the people than about the model output. We want it to be true, so we squint and connect dots and it's true. But it isn't. It's math and tricks, and if human intelligence is truly nothing more than math and tricks, then what we have today is a tiny, tiny, tiny fraction…

It’s the stupid term AI that ruined everything. Our predecessors had enough intelligence, ironically, to call spellcheck what it is. Today it would be AI.

Re: Chatbots: Still dumb after all these years

#264

Post by edflsafoiewq »

emtel wrote: post #239
amiantos wrote: post #234

Earlier quoted context omitted.

What's impressive about this conversation? I don't feel like it is very complicated at face value. You're telling to program to start pulling facts from the life of a public figure, and then it does so. You can get the same out of Google.

Read the last two lines again. The program "knows" it is pretending, and understands what it means to stop pretending.

It is interesting, but the extent of "pretend" is just to swap "Ada Lovelace" and "I". I wonder what would happen if you asked "What do you know about Alan Turing?"

Re: Chatbots: Still dumb after all these years

#265

Post by masswerk »

klyrs wrote: post #12

My favourite flaw of chatbots is exposed by ELIZA. Not chatting with ELIZA, (though, it does suffer this flaw) but using responses inspired by that program. "Please elaborate on that" or "tell me more about [noun]" etc. Bots appear to have zero lines of short term memory, and utterly fail to pick up a reference to the thing that they just said. My favorite being bot> [something plausibly human-sounding] me> What do y…

For the record, ELIZA does have a short term memory, but it is only used as a fallback, when other rules fail to match. (E.g., in productions like, "Earlier you said *" or "What else comes to your mind when you think of *" by the famous DOCTOR rules set.)

ELIZA/DOCTOR manages somewhat to succeed with this, because it pretends a setting, where there are no outside references at all, beyond the very limited purpose of carrying on with a conversation that is meant to explore internal subjects in ever-shifting context. (You could say, the shift in context is the actual progress and success of the conversation, while having to pick up an earlier thread actually marks a point of failure.) Outside of this restrictive setting, the shortcomings of the absence of a consistent world model become obvious very soon.

Re: Chatbots: Still dumb after all these years

#266

Post by Maxion »

repomies69 wrote: post #231
klabb3 wrote: post #195

Earlier quoted context omitted.

> Anyone who questions or points out that the technology doesn't do what the business people think it does[...] Uh oh, we've got a downer! Jokes aside, I'd like to consider an even simpler explanation, namely that "The purpose of a system is what it does"[1]. In this case, it would suggest decision makers are fully aware that they suck. Why would anyone want something that sucks? Because it's discouraging, and custom…

The chatbot saves money. Simple as that. People get served by chatbots, get frustrated, and majority gives up and doesn't bother the company with the problem. It doesn't really matter how the chatbot saves the money, they can just see the end results and use the money for bonuses instead.

>People get served by chatbots, get frustrated, and majority gives up and doesn't bother the company with the problem.

With customer service systems I have to disagree here. I've got experience working with these issues and for the vast majority of companies most contacts to customer support are for small, banal, things that current crappy chatbots can easily solve. The 80-20 rule works here too.

Bad companies try to do too many things with their chatbots, and use it as a way to make it harder for customers to actually get to talk to a human.

Good companies use them to allow customers to solve common issues faster than getting a real human on the line.

Then there's the other issue of how people want to find information - some people want to find it themselfes and do not want to talk to a person, whereas others specifically want to ask someone and do not want to look at all. For the latter group, no chat bot or fancy autocomplete knwoledgebase (Looking at you Zendesk) or similar will work. They'll always try to contact support as their first step, and they will get very frustrated if you make it impossible.

If your a more software oriented company, it can be very benificial to have software developers (or even a team, if you're a larger company) devoted to helping support. The majority of issues customers contact support for are UI/UX or business process related, and if you systematically start to solve them you will reduce customer support burden and customer support related costs.

Re: Chatbots: Still dumb after all these years

#267

Post by davidhariri »

I don’t disagree with the general “this is taking longer than we expected” premise, but the author hasn’t tuned GPT-3 properly. I don’t know if that was on purpose or not, but I have seen GPT-3 answer questions about presidents accurately with the right tuning.

The point on facts still stands though. To my knowledge, transformer models aren’t great at knowing how factual their predictions are which is certainly an obstacle to business adoption. They also tend to overfit, in my experience.

Re: Chatbots: Still dumb after all these years

#269

Post by visarga »

pdimitar wrote: post #252
visarga wrote: post #248

Earlier quoted context omitted.

> Michele Obama classified as a male Yes, humans don't make embarrassing mistakes all the time. We're only making dignified mistakes.

If something is to replace humans at doing a job X then it has to be better than the humans at job X. Also humans instinctively expect the machines to not make mistakes. I haven't made them such; it's just a fact of life. And we have to work with the expectations of the bigger populace.

It's not necessary to be perfect, just useful.

Re: Chatbots: Still dumb after all these years

#270

Post by Traster »

emtel wrote: post #239
amiantos wrote: post #234

Earlier quoted context omitted.

What's impressive about this conversation? I don't feel like it is very complicated at face value. You're telling to program to start pulling facts from the life of a public figure, and then it does so. You can get the same out of Google.

Read the last two lines again. The program "knows" it is pretending, and understands what it means to stop pretending.

No, read the conversation again, it's not pretending. It's reciting facts from wikipedia but conjugating in the first person instead of the third.

>User What are your parents like?

My dad is a celebrity, he's a poet, he's a larger than life figure. I didn't know him well, he didn't treat my mother well. I grew up with my grandmother and didn't really spend much time with my father.

For a human, when you say "what are you parents like" it illicits a thousand more things than just repeating their official titles. Read the Byrons' wikipedia pages, and think how you would answer that question.

Post reply on HN