Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chatbots: Still dumb after all these years

mindmatters.ai

Post reply on HN
Print view View original

341–350 of 426 posts

Re: Chatbots: Still dumb after all these years

#341

Post by aeternum »

Iv wrote: post #338
pwinnski wrote: post #13

Having worked in ML at two different companies now, I think that people interpreting model output as intelligence or understanding says much more about the people than about the model output. We want it to be true, so we squint and connect dots and it's true. But it isn't. It's math and tricks, and if human intelligence is truly nothing more than math and tricks, then what we have today is a tiny, tiny, tiny fraction…

I usually sum it up by saying that what we have is a very good bullshit generator. It is bad at reasoning but good at making up answers that sounds correct and crossing fingers that the corrector won't look twice at what they say.

This is true but I think we also overestimate how good humans are at reasoning. Throughout the vast majority of human history we simply made up answers that sounded correct.

The scientific method and the importance of truth is a relatively recent development.

Re: Chatbots: Still dumb after all these years

#342

Post by JetAlone »

andrewla wrote: post #152

The ending is the relevant bit: > As I have said many times, the real danger today is not that computers are smarter than us, but that we think computers are smarter than us and consequently trust them to make important decisions they should not be trusted to make.

I think the biggest risk vector taking us along that conga line is poor mental health. Vulnerable people feel weak and helpless enough to actually believe they ought to depend on on a system to take the work/responsibility out of their hands.

Re: Chatbots: Still dumb after all these years

#343

Post by hugh-avherald »

ShinTakuya wrote: post #332
foolfoolz wrote: post #291

Earlier quoted context omitted.

don’t generalize these things. self driving in every possible scenario yes is probably a while off. self driving in specific circumstances is already here, such as airport shuttles and moving shipping containers in docks. you don’t need to solve the general case to be useful

This. And in fact I think that may be the path forward - just update all the main roads to be self driving friendly, like a train network, and keep the smaller roads as manual driving only.

"just update"

Re: Chatbots: Still dumb after all these years

#344

Post by mike00632 »

Majestic121 wrote: post #70
mike00632 wrote: post #47

Earlier quoted context omitted.

What if that's the case? What if me replying to you is nothing more than some atomically basic mechanism of statistical inference and randomness? Your entire perception of my personhood could be based on a sense of verisimilitude without significant reasoning to set me apart from a transcendant spirit. What then? Would you be less forgiving of my mistakes if you knew? Would you be less proud of work we collaborate on…

I think like OP that it's mostly math and tricks, and that we're far from reaching it yet. But then, the conclusion is not to be less forgiving, but more forgiving : we're all just meat machines, we should be forgiving toward each other's flaw (not naive mind you, but forgiving), and proud of what we reach collectively with our lowly condition.

I mean, what if we're all math and tricks, i.e. that consciousness is an illusion. We might be chasing an "it" that doesn't exist. In a more practical sense, what I'm getting at is that we might be applying an arbitrary or undefined standard to AI progress, especially when we compare it to people.

Re: Chatbots: Still dumb after all these years

#345

Post by otterley »

Business that don't actually provide the underlying mechanisms to make customers happy -- for example, an API that refunds your money -- aren't going to be able to slap a chatbot in front of it that makes customers happy, no matter how "smart" the chatbot is. If the underlying problem cannot be solved, all the business is doing at best is providing an obfuscation layer in front of it.

And if the problem can be solved via an API mechanism, you don't need that smart a chatbot to get the job done: word-matching is often enough. "Refund", or "reschedule," or "where's my money?" followed by a step-by-step series of button-pushing that takes the customer through the process often does the job just fine.

Re: Chatbots: Still dumb after all these years

#346

Post by gitgud »

> "As I have said many times, the real danger today is not that computers are smarter than us, but that we think computers are smarter than us and consequently trust them to make important decisions they should not be trusted to make."

This is the crux of AI in general today, companies promote cherry-picked examples from their AI system and people get a distorted view of what the AI is capable of.

Re: Chatbots: Still dumb after all these years

#347

Post by katmannthree »

mcswell wrote: post #339
not2b wrote: post #157

Earlier quoted context omitted.

No, they still lack the common sense of a toddler, because they don't know anything about the world; they only know (in great detail) about the structure of their training data.

Apparently they (or at least the person you were responding to was "talking" to) also "know" about data they find on the web, which does not need to be in their training data. There was a news article recently about an 8(?) year old who asked Alexa for a "challenge." "Plug a charger part way into an electric outlet," replied the Alexa", "and touch a penny..." (you can guess the rest--I won't put it here in case anoth…

I don't believe GPT-3 (which is what I was using and commenting on) has access to the internet. It was trained on a large data set from the web, but for instance if you ask it about COVID-19 it has absolutely no idea what you're talking about.

Alexa is a different matter entirely, that one actively troll the web in response to your queries.

Re: Chatbots: Still dumb after all these years

#348

Post by runarberg »

foolfoolz wrote: post #291
Traster wrote: post #273

This is essentially why I think we're not going to get self-driving. A few years ago we produced something that could do a good job of pretending to drive under a tiny number of circumstances, and people went "OMG it can drive!" and assumed that we just have a few corner cases before the chat bot can have a conversation about anything. The truth is that often these very limited solutions are just that - limited, they…

don’t generalize these things. self driving in every possible scenario yes is probably a while off. self driving in specific circumstances is already here, such as airport shuttles and moving shipping containers in docks. you don’t need to solve the general case to be useful

pfft, we’ve had self driving trains for more then half a century. What you are describing is basically that with more fancy sensors instead of the rails, and perhaps the rails are probably a superior guidance technology in most of these cases anyway... unless the purpose is to impress rather then to automate.

Re: Chatbots: Still dumb after all these years

#349

Post by barthelomew »

thewarrior wrote: post #286
version_five wrote: post #42

Earlier quoted context omitted.

I think a problem is the tighter cycle between academic discoveries and business people trying to monetize them. Large language models were developed, objectively a great achievement, and immediately someone nontechnical wants to apply their own interpretation and imagine that we can build a chatbot that you won't have to pay, and before you know it, people are selling and even deploying them. Anyone who questions or…

GPT-3 needs proper prompting to get the right output. Here's the same question as in the blogpost posed differently Student: Why is it unsafe to walk backwards down a staircase with my eyes closed ? Teacher: You can't see the step behind you! The "teacher" here is GPT-3 and answered the question easily. People are in denial of genuine progress. Student: Do you think I could predict changes in stock prices better if I…

Once you start prompt-engineering, the answers, in seclusion look pretty great - and show some understanding of the domain.

Yet, if you donot do this - they end up spitting out some randomly associated phrases/answers. This is a problem when you're asking a model, a question who's answer you donot completely know. How do you trust it to give the right answer?

If you donot know the answer beforehand - you cannot prompt-engineer the model to give the "right answers".

"Expert systems" from the 80s and the 90s, were pretty good at giving the "right answers" inside a closed domain. Those were truly wonderful systems.

Re: Chatbots: Still dumb after all these years

#350

Post by GrantS »

thewarrior wrote: post #286
version_five wrote: post #42

Earlier quoted context omitted.

I think a problem is the tighter cycle between academic discoveries and business people trying to monetize them. Large language models were developed, objectively a great achievement, and immediately someone nontechnical wants to apply their own interpretation and imagine that we can build a chatbot that you won't have to pay, and before you know it, people are selling and even deploying them. Anyone who questions or…

GPT-3 needs proper prompting to get the right output. Here's the same question as in the blogpost posed differently Student: Why is it unsafe to walk backwards down a staircase with my eyes closed ? Teacher: You can't see the step behind you! The "teacher" here is GPT-3 and answered the question easily. People are in denial of genuine progress. Student: Do you think I could predict changes in stock prices better if I…

Yes, this is both the most misunderstood and the most amazing (and unpredicted?) thing about GPT-3 et al. You have to tell it who it is and what the rules are. And it gains or lacks knowledge depending upon who it thinks it is answering as. You’ll find that Stephen Hawking doesn’t have much to say about the Ninja Turtles while celebrities know nothing about black holes. It does not answer questions or have expertise unless you tell it that it is answering questions and has expertise. (And it is incredible that this is the case, and somewhat understandable that people don’t understand this.)
Post reply on HN