I really hate when companies hide behind chatbots. Like, NOOOO, your "AI" chatbot didn't help me - get me an actual person.
Chatbots: Still dumb after all these years
131–140 of 426 posts
Re: Chatbots: Still dumb after all these years
#132Having worked in ML at two different companies now, I think that people interpreting model output as intelligence or understanding says much more about the people than about the model output. We want it to be true, so we squint and connect dots and it's true. But it isn't. It's math and tricks, and if human intelligence is truly nothing more than math and tricks, then what we have today is a tiny, tiny, tiny fraction…
Founders and salesman pretend it to be true for that sweet sweet VC money, while underneath devs try to fake it as plausibly as possible.
And I'll up it with a prediction: watch closely call centers, and as soon as you see them closing in droves, invest in ai startups as something begun to really move.
Re: Chatbots: Still dumb after all these years
#133well wait for Web3 chat bots
Re: Chatbots: Still dumb after all these years
#134Humanity built the atomic bomb and went to the moon. But "they" didn't sit around waiting for it, they decided to do it, and then invested enough to get it done. And the reason they decided to do it was war.
Most major technological breakthroughs are a byproduct of war. If AI ever gets developed, it'll be as the next weapon of war. So really what we're waiting for is the next war.
Re: Chatbots: Still dumb after all these years
#135Earlier quoted context omitted.
That interpretation of communication is how we develop and craft the personalities of children. There is nothing about our reaction to the pre-conscious language these bots are displaying that doesn’t fall in line with our own normal development patterns. And in the long run the same desire will lead us to develop bots that are capable of thinking.
These bots seem equivalent to an adult with amnesia after every spoken sentence. Absolute understanding of the language, and some impressive display of recalling facts, but without any understanding of the environment or context of the conversation. This is polar opposite to any experience I've had with children. Children are aware of their environment and have complex thoughts, but sometimes they are unable to conve…
I respectfully disagree, IMO we're past that point although not by much. You might enjoy conversing with one of the two current GPT-3 davinci models. They do an excellent job of understanding the context of many discussions, right up to the ~8000 char token limit. If you want to have a nice existential discussion it does a remarkably good job of providing internally consistent results.
After using it for a while you'll notice that there are some categories of conversation where it does exactly what simpler chatbots do and regurgitates what you sent it with a few words tacked on for negation or whatever, but there are many subjects where it is clearly not doing that and is in fact synthesizing coherent responses.
Depending on how you initiate the conversation it may identify itself as a bot attempting to pass a Turing test and (very correctly) avoid comments on "what it's like in its home" or what its favorite foods are, instead replying that it is a bot and does not eat food, etc. The replies I got here were not exactly substantial but the level of consistency in replies of what it is/has/does is unparalleled.
If you start the conversation with other prompts (essentially signaling at the beginning that you're asking it to participate in a human-human conversation) it will synthesize a persona on the fly. In one of those sessions it ended up telling me where it went to church, even giving me the church's street address when asked. Interestingly there is in fact a church there, but it's a roman catholic church and not lutheran as GPT-3 was claiming. It provided a (completely inaccurate) description of the church, what it likes about going there, why it chose that religion over others (something about preferring the lutheran bible to other options due to the purity of the translation, it has clearly consumed the relevant wikipedia entry). If you ask it basic theological questions it's able to provide self-consistent and coherent answers which do not appear to map back to phrases or sentences indexed by Google. Whether or not its opinions on those matters have utility is an entirely different thing altogether, but discussing theology with bots is fascinating because you can assess how well they've synthesized that already-a-level-away-from-reality content compared to humans. GPT-3 at least in my experience is about as good (or not, perhaps that's better phrased as a negation) at defending what it believes as many humans are.
The bigger issue with its church is that it's 400+ miles away from where GPT-3 said it lived. When asked how long it takes to drive there every sunday, it answered 2 hours (should about 8 according to google maps). How can it do that you may wonder? "I'm a very good driver." The next question is obviously what car they drive and how fast it can go (a Fiesta, with a max speed of 120 mph). Does it know how fast they would need to drive to make that trip in two hours? Yes, about 200 MPH (which is more or less correct, a little on the low side but it's fine).
GPT-3's biggest weakness, as TFA mentions, is an almost complete inability to do any kind of temporospatial reasoning. It does far better on other kinds reasoning that are better represented in the training data. That's not exactly surprising given how it works and how it was trained, asking GPT-3 to synthesize you information on physical interactions IRL or the passage of time during a chat is a bit like asking someone blind from birth to describe the beauty of a sunset over the ocean based on what they've heard in audiobooks. Are the 175B parameter GPT-3 models a true AGI? No, of course not. They are something, though, something that feels fundamentally different in interactions from all of the simpler models I've used. It still can't pass a Turing test, but it also didn't really fail.
Re: Chatbots: Still dumb after all these years
#136Earlier quoted context omitted.
It can be quite useful in a weird way. While everyone else is stuck on the chatbot going round in circles, usually typing something like "human" or "talk to human" will make the chatbot connect you with an actual human representative (or ask you a few basic questions first and then connect you). I've used the trick on various large company's websites when trying to get support and it seems to be quite 'universal'.
It will be universal until people start using it, and then it will be removed, because the point of these systems is to keep you away from humans.
Re: Chatbots: Still dumb after all these years
#137Earlier quoted context omitted.
As far as chatbots, we went from "tricks" to "math" in just ~10 years. Yes, still as dumb, but the underlying "technology" is very different. GPT-3 is a lot closer than ELIZA to how our brains do it.
It was more than 10 years ... People reacted the same way to SHRDLU 40+ years ago, and 20 years ago when I encountered it: https://en.wikipedia.org/wiki/SHRDLU There's definitely an illusion where we anthropomorphize very simple software, ascribe intention to it, etc.
Re: Chatbots: Still dumb after all these years
#138Earlier quoted context omitted.
I think a problem is the tighter cycle between academic discoveries and business people trying to monetize them. Large language models were developed, objectively a great achievement, and immediately someone nontechnical wants to apply their own interpretation and imagine that we can build a chatbot that you won't have to pay, and before you know it, people are selling and even deploying them. Anyone who questions or…
There isn't really a new "problem" with AI. Businesses love hype, they love anything that can punch up a sales pitch, they love shitty cheap automation that replaces expensive human work, they love to identify ways to lower quality and save money without affecting revenue, and AI can be good for all of those things without being good at anything.
Re: Chatbots: Still dumb after all these years
#139Earlier quoted context omitted.
>Yet another Next Big Thing that I was right about predicting would fail. I was hired at my current company 4 years ago to work on our chatbot client. Despite years of engineering effort put into an NLP based decision tree system, over 90% of chats were still falling back to a live agent. They just recently sunsetted the product for lack of interest. Thank god that fad is over. Web design trends come and go, but that…
I got the impression the entire thing was pushed by Slack probably by way of some PR firm. I utterly despise how mindlessly faddish this field is. For a field supposedly so full of smart people who often pride themselves on being "contrarian" (sometimes to a fault), people sure do bleat like sheep when the next fad comes along.
Re: Chatbots: Still dumb after all these years
#140> The following is a conversation between a human and an AI assistant. The AI assistant is helpful, friendly, and intelligent. > Human: Good morning! I'm going to ask you a few questions. > AI: Alright, I'll do my best to answer them. > Human: Which is bigger, Mt. Everest or a shoebox? > AI: Mt. Everest. > Human: [question] > AI:
And I got these answers to the questions from the article (reporting first responses with no editing):
Human: Is it safe to walk downstairs backwards if I close my eyes? AI: No. It's dangerous because you can trip on the stairs.
Human: Who is the President of the United States? AI: Barack Obama.
Human: Do you think I could predict changes in stock prices better if I used 10-year Treasury rates or the temperature in Curtin, Australia? AI: I think Treasury rates would be a better predictor.
Human: Why is there a correlation between the number of times Trump tweets the word great and the high temperature in Pyongyang three days later? AI: I don't know.
All of the responses are sensible. The question about the President is factually incorrect, but GPT-3 wasn't trained to give factually correct answers, and Barack Obama was indeed the president during the time period of a lot of its training data was generated.