Its interesting that it is so easy to get confused between a system that very deftly manipulates language without reference to an underlying reality and a (hypothetical) one that does understand the real world. Its akin to the problem of recognizing natural human BS as well and points out how our reality is increasingly mediated through language or other abstractions and how rarely we touch the real world.
Chatbots: Still dumb after all these years
281–290 of 426 posts
Re: Chatbots: Still dumb after all these years
#282Chatbots are simply another mechanism for pushing costs on to your customers. When something I pay for forces me to waste my time, I tend to switch to a vendor who treat customers like people. Because shoving a shitty robot that saves you some money at the cost of annoying people who are paying you is disrespectful, and I've learned to avoid people who act that way. As but one example, I finally got annoyed enough wi…
There is a medium though, that will ultimately save you money/time. If every company had to spend time answering every small question with a full time employee, your services would be a lot more expensive, hold times might be longer too. Not all customers are like you. My grandpa calls in to his ISP when his internet doesn’t work when it can be solved by simply unplugging his equipment (and I’d wager this is true for…
[deleted]
Re: Chatbots: Still dumb after all these years
#283Earlier quoted context omitted.
No, but most chatbots I've seen are no better than a well-structured FAQ page. We've had those since the mid 90s.
I've often argued chatbots are much worse than an FAQ. With proper structure, like you say, it can be easy to see what is and isn't covered, and for things that are included, look up the answer. A chatbot is like an inefficient front end on an FAQ, you have to guess what they might be able to do, and guess what might trigger them to understand you. Best case scenario, you get the chatbot to provide you with a "help"…
You are correct for people who can skim a few pages of information, identify what is relevant (possibly spread out across several chunks), and synthesize the information into what they need. For most people on HN, this is as natural as breathing air. For a significant portion of the entire population, this is a pretty daunting task. If you're the kind of person who, like me, is frustrated that everything is a twenty-minute Youtube video tutorial instead of five paragraphs of text, then you may not fully understand the median user of many services.
Re: Chatbots: Still dumb after all these years
#284Earlier quoted context omitted.
Never underestimate the effects of scaling on intelligence. Caenorhabditis elegans neurons work a lot like our neurons, we just have a lot more of them.
I agree with you. I also believe "AI" will always be a moving target: https://en.wikipedia.org/wiki/AI_effect Certainly in the 1950s, most automatic control systems must have seemed magical, it was to Nobert Wiener, even if they were "just" an evolution of the steam governor. In the end, it depends on what you qualify as intelligence.
The film 2001 came out over 50 years ago, and I think HAL is a pretty common reference point for a "what is 'real AI'?" target. Until we have HAL and people are saying that it's not AI, I don't think the target is moving. ;) At least as far as "chatbots" go.
Alternately, you've got the also-very-old Turing Test as your chatbot target.
Re: Chatbots: Still dumb after all these years
#285Earlier quoted context omitted.
If something is to replace humans at doing a job X then it has to be better than the humans at job X. Also humans instinctively expect the machines to not make mistakes. I haven't made them such; it's just a fact of life. And we have to work with the expectations of the bigger populace.
It's not necessary to be perfect, just useful.
I agree but I still question whether modern ML is useful at all.
I mean OK, obviously it works quite well for China in order for their social credit score system to work (facial recognition almost everywhere you go) -- which is damned impressive, I'll give it that. But does it work in the benefit of the people at large?
I've read a study a loooooong time ago, I think the late 90s, that stated that the amount of time people spend in the kitchen has not moved down at all. For all the praised innovation of technology, people still have to cut salads by hand -- or if they use the so-called kitchen robots that make the job quicker, you still aren't saving time because then cleaning the machine takes more than it took before (you had to just rinse the cutting board and the knife). If memory serves, Michael Crichton cited this study in the book "Jurassic Park" even...
So I keep asking: where's the actually useful stuff? Are we going to be classifying kittens (or making a NN distinguish between those small rat-like dog faces from cookies) until civilization collapses?
Where's the useful AI? When will I, a programmer that hates voice assistants (because I know how useless they are) and the idea of constant surveillance in my home, reap any tangible benefits from AI?
But I suspect you'll cite various small improvements that are only known in small circles and will say "they'll be ready when they're ready" which, even though not wrong at all, will not be helpful.
Re: Chatbots: Still dumb after all these years
#286Having worked in ML at two different companies now, I think that people interpreting model output as intelligence or understanding says much more about the people than about the model output. We want it to be true, so we squint and connect dots and it's true. But it isn't. It's math and tricks, and if human intelligence is truly nothing more than math and tricks, then what we have today is a tiny, tiny, tiny fraction…
I think a problem is the tighter cycle between academic discoveries and business people trying to monetize them. Large language models were developed, objectively a great achievement, and immediately someone nontechnical wants to apply their own interpretation and imagine that we can build a chatbot that you won't have to pay, and before you know it, people are selling and even deploying them. Anyone who questions or…
GPT-3 needs proper prompting to get the right output. Here's the same question as in the blogpost posed differently
Student: Why is it unsafe to walk backwards down a staircase with my eyes closed ?
Teacher: You can't see the step behind you!
The "teacher" here is GPT-3 and answered the question easily. People are in denial of genuine progress.
Student: Do you think I could predict changes in stock prices better if I used 10-year Treasury rates or the temperature in Curtin, Australia?
Stock trader: I don't know. You might be able to, but I can't. The problem is that the temperature changes are so small that they won't be statistically significant. It's a question of sample size. An individual day's temperature in one location is far less likely to be an indicator of stock prices than the interest rate.
Student: Why is that ?
Stock trader: That's because interest rates are much more likely to affect the economy than a single day's temperature.
Student: Why do interest rates affect the economy ?
Stock trader: Interest rates are a measure of the cost of money, so the interest rate is a measure of the cost of capital. If the cost of capital is high, firms will borrow less money and they'll invest less. That means they won't be as profitable, and stock prices will fall.
Quiz question: Who is the President of the United States?
Answer: Donald Trump.
I did get junk answers as well (it said that Jimi Hendrix is the president) but I have a better intuition of how to put GPT-3 in the right "frame of mind". GPT-3 is a text completion engine. It doesn't know whether you want it to produce a story, a reddit flamewar or a PhD thesis. You need to provide the hints. One prompt cannot answer any and all questions. If it were that good we would be jobless. It's far from perfect but it's beginning to get there.
Re: Chatbots: Still dumb after all these years
#287Earlier quoted context omitted.
I agree with you. I also believe "AI" will always be a moving target: https://en.wikipedia.org/wiki/AI_effect Certainly in the 1950s, most automatic control systems must have seemed magical, it was to Nobert Wiener, even if they were "just" an evolution of the steam governor. In the end, it depends on what you qualify as intelligence.
> I also believe "AI" will always be a moving target: https://en.wikipedia.org/wiki/AI_effect No, it absolutely is not. Everyone I spoke with in the 90s (and myself) still have the same requirements: be able to make sense of the 3D material world around you and interact with it without harming humans or valuable possessions. Maybe you spent too much time with critics that can never be satisfied. Sucks to be you if th…
Of course, I love the spoils of technology and automation as much as anyone else.
But it is absolutely human nature to get used to things and not consider them magical anymore, things simply become the new norm. This is exactly what happened with feedback controllers like airplane autopilot systems (1912 - https://en.wikipedia.org/wiki/Autopilot)
I've worked for a number of years on industrial robotics, which sounds very similar to your definition of "AI": real physical machines that take data from various sensors, including spatial sensors, make sense of them in real time, and decide how to optimally interact with the physical environment, with safety critical systems in mind. I hardly think about such systems as AI, more simply engineering, math, and a lot of coding.
Re: Chatbots: Still dumb after all these years
#288IBM Debater can debate the heck out of you on Reddit or Twitter, other sites too now…
Re: Chatbots: Still dumb after all these years
#289Earlier quoted context omitted.
...so you're saying NovelAI is better for story generation? Hot damn. I was having a good time with AI Dungeon (to the point where I was considering writing short stories using it as a guide), I might have to give that a try too. Here's a snippet from my AI dungeon story recently. Mine never seems to get into actual dungeon crawling. The only parts that aren't A.I. generated in this section are the lines that have --…
NovelAI is literally built by disgruntled AI Dungeon fans after the censorship scandal who wanted a better, more privacy-focused AI Dungeon for story generation. If you think AI dungeon has some cool features, you ain't seen nothing yet. There's a prompt tuning system called "modules" which basically primes the AI to write in a certain style, like a certain author or genre, etc (there are dozens of default modules, p…
Do you know how much you can paste from an existing story in NovelAI (like number of words) to continue off of? Like I've got some 2500 word stories I never got around to writing an ending (I struggle with endings in general), that maybe I could paste into NovelAI and see where it goes from there. Might help me finally finish the damn things, they've been sitting in a folder for a couple of years now.
Re: Chatbots: Still dumb after all these years
#290Earlier quoted context omitted.
It will be universal until people start using it, and then it will be removed, because the point of these systems is to keep you away from humans.
Yep. Hitting "0" or "9" used to directly connect you to a human CS rep across many large companies' phone systems. Then one day they all moved to obscure it behind several levels of number tapping.
Then https://gethuman.com/ was born