Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chatbots: Still dumb after all these years

mindmatters.ai

Post reply on HN
Print view View original

181–190 of 426 posts

Re: Chatbots: Still dumb after all these years

#181

Post by version_five »

semi-extrinsic wrote: post #138
dkarl wrote: post #100

Earlier quoted context omitted.

There isn't really a new "problem" with AI. Businesses love hype, they love anything that can punch up a sales pitch, they love shitty cheap automation that replaces expensive human work, they love to identify ways to lower quality and save money without affecting revenue, and AI can be good for all of those things without being good at anything.

No, but most chatbots I've seen are no better than a well-structured FAQ page. We've had those since the mid 90s.

I've often argued chatbots are much worse than an FAQ. With proper structure, like you say, it can be easy to see what is and isn't covered, and for things that are included, look up the answer.

A chatbot is like an inefficient front end on an FAQ, you have to guess what they might be able to do, and guess what might trigger them to understand you. Best case scenario, you get the chatbot to provide you with a "help" response that's basically the FAQ.

A simple list of options will always beat a "conversational interface" to a list of potential back-end actions.

Incidentally, I think this gets obscured a but when dealing with businesses whose goal is to confuse or frustrate you into not taking action. If you look at Amazon or Uber's FAQ, they are designed to take you in circles and prevent you from getting any recourse or talking to a person. Chatbots could be helpful for this application

Re: Chatbots: Still dumb after all these years

#182

Post by namdnay »

brettdeveffect wrote: post #129
banana_giraffe wrote: post #127

In the early days I worked at a company that had a natural language chatbot product. It wasn't an online thing, but rather part of a larger tool. You could ask it to do things like "show me the quarterly spreadsheet", and if it didn't know what "quarterly spreadsheet" was, it'd ask questions in English and learn what "quarterly" and "spreadsheet" meant. And it could use that new knowledge to update its questions so i…

What you are describing counts as ML.

Yes it’s literally a machine that learns!

Re: Chatbots: Still dumb after all these years

#183

Post by fault1 »

JamesBarney wrote: post #161
fault1 wrote: post #148

Earlier quoted context omitted.

I remember chatbots in 2000 using hidden markov models (and sometimes even neural nets), so from that standpoint, they aren't exactly new. The models are just larger now.

Never underestimate the effects of scaling on intelligence. Caenorhabditis elegans neurons work a lot like our neurons, we just have a lot more of them.

I agree with you.

I also believe "AI" will always be a moving target: https://en.wikipedia.org/wiki/AI_effect

Certainly in the 1950s, most automatic control systems must have seemed magical, it was to Nobert Wiener, even if they were "just" an evolution of the steam governor.

In the end, it depends on what you qualify as intelligence.

Re: Chatbots: Still dumb after all these years

#184

Post by AshamedCaptain »

klyrs wrote: post #30
hotpotamus wrote: post #23

Earlier quoted context omitted.

I'm not trying to be mean or break any HN rules, but did you read the article? It basically covers what you asked and was quite a revelation to me. Others in here point out that GPT-3 is not a chatbot which is good info, but I also wonder if there is anything out there that can even "remember" the context of a conversation it's supposed to be having. Particularly interesting is the question from the article, "who is…

Hidden in my comment is a question: has anybody even tried to included previous lines into the prompt, and does that not help? Asking here because there are a lot of ML nerds who could probably do a proof of concept in a few hours, if not a few lines of code, so there's a decent chance I'll get an informed response.

A lot of chatbots from 20 years ago did that. They would even use it as a escape route, and would just bring back the previous topic of conversation (from say 10 lines ago) when they could not figure out something to come up for the current prompt. It really didn't help the illusion; rather it was even more obvious.

Re: Chatbots: Still dumb after all these years

#185

Post by soco »

semi-extrinsic wrote: post #138
dkarl wrote: post #100

Earlier quoted context omitted.

There isn't really a new "problem" with AI. Businesses love hype, they love anything that can punch up a sales pitch, they love shitty cheap automation that replaces expensive human work, they love to identify ways to lower quality and save money without affecting revenue, and AI can be good for all of those things without being good at anything.

No, but most chatbots I've seen are no better than a well-structured FAQ page. We've had those since the mid 90s.

Most? I'd be surprised to see a single one working beyond the FAQ level (a very appropriate comparison, thank you).

Re: Chatbots: Still dumb after all these years

#186

Post by beepbooptheory »

sam0x17 wrote: post #143
melony wrote: post #48

Earlier quoted context omitted.

That's a good thing. The market is very good at empirically validating research. It helps filter out the chaff and ensure research is useful. Better than academia wasting years on concepts like the "semantic web" where nobody can even agree on what it means. Academia is under attack from many political directions right now, being able to show useful output will help it thrive in the long run.

Capital is far too short-sighted to look beyond this year or this quarter. Many of the important groundbreaking ideas or projects take years, even decades to formulate. As I've said elsewhere, capital will just take the easiest road it is allowed to take, which currently is stifling innovation, buying competitors, maintaining monopolies, relying on economies of scale to shut out competitors when they can't be acquire…

This is a really great way to frame it, because it really does seem to be about time/speed. I don't think we can even collectively comprehend great innovations that need many resources and time anymore, who has even the time to work on it? People need to have jobs after all.

Re: Chatbots: Still dumb after all these years

#187

Post by dragonmost »

300bps wrote: post #44
pwinnski wrote: post #13

Having worked in ML at two different companies now, I think that people interpreting model output as intelligence or understanding says much more about the people than about the model output. We want it to be true, so we squint and connect dots and it's true. But it isn't. It's math and tricks, and if human intelligence is truly nothing more than math and tricks, then what we have today is a tiny, tiny, tiny fraction…

We want it to be true, so we squint and connect dots and it's true. I've had numerous people look at the actions of software I wrote over the last 30 years make comments like, "Ohhhhhh it probably did that because it knew x". The software had no such functionality. Seems to be a natural thing for people to be overly optimistic about software's capabilities.

In the game speedrunning community it's impressive the number of time a glitch or behavior of the game is attributed to "the game is confused".

Re: Chatbots: Still dumb after all these years

#188

Post by [deleted] »

300bps wrote: post #64
Sharlin wrote: post #35

Earlier quoted context omitted.

The GPT2 version of the AI Dungeon [1] could keep track of context for maybe couple of lines at a time. I've heard the GPT3 version is substantially better. The problem is, of course, that these "AI" chatbots on websites, marketing buzzwords notwithstanding, have very little to do with state-of-the-art machine learning, and are indeed unlikely to be any more sophisticated than the original ELIZA for the most part. [1…

Thanks for the link to that! I tried it out just now and I have to say that I am not impressed. What will you do? You get longsword from the ground. You get your longsword from the ground. You hold it in your hands and examine it for a moment. It's a very sturdy weapon. The knight then grabs his longsword and bows. Huh? Why did it grab the bow as well? You inventory. You have a crossbow. Huh? I thought I just grabbed…

[deleted]

Re: Chatbots: Still dumb after all these years

#189

Post by dane-pgp »

hotpotamus wrote: post #23
klyrs wrote: post #12

My favourite flaw of chatbots is exposed by ELIZA. Not chatting with ELIZA, (though, it does suffer this flaw) but using responses inspired by that program. "Please elaborate on that" or "tell me more about [noun]" etc. Bots appear to have zero lines of short term memory, and utterly fail to pick up a reference to the thing that they just said. My favorite being bot> [something plausibly human-sounding] me> What do y…

I'm not trying to be mean or break any HN rules, but did you read the article? It basically covers what you asked and was quite a revelation to me. Others in here point out that GPT-3 is not a chatbot which is good info, but I also wonder if there is anything out there that can even "remember" the context of a conversation it's supposed to be having. Particularly interesting is the question from the article, "who is…

> this is a common question paramedics will give to people who have suffered head injuries to assess consciousness.

I can't help wondering what the assessment would be if someone answered "Who is the current president?" with "Watson Bakery".

https://www.youtube.com/embed/cS1Tmz816hk

Re: Chatbots: Still dumb after all these years

#190

Post by dangero »

version_five wrote: post #181
semi-extrinsic wrote: post #138

Earlier quoted context omitted.

No, but most chatbots I've seen are no better than a well-structured FAQ page. We've had those since the mid 90s.

I've often argued chatbots are much worse than an FAQ. With proper structure, like you say, it can be easy to see what is and isn't covered, and for things that are included, look up the answer. A chatbot is like an inefficient front end on an FAQ, you have to guess what they might be able to do, and guess what might trigger them to understand you. Best case scenario, you get the chatbot to provide you with a "help"…

yes chat bot interface discoverability is terrible -- in fact it feels like the system you would design if the intended goal was to intentionally obscure the interface surface while still allowing access to it

One logical reason for obscuring the real interface is to make the interface appear more robust than it actually is, so pure marketing, but outside of that I'm not sure what the value is.

similar to home assistants Siri/Alexa/Google where 99% of the creative things you try don't work so you end up with just using it to set timers and play music

Post reply on HN