Live data from Hacker News

Chatbots: Still dumb after all these years

mindmatters.ai

241–250 of 426 posts

Re: Chatbots: Still dumb after all these years

#241
post #30

Earlier quoted context omitted.

I'm not trying to be mean or break any HN rules, but did you read the article? It basically covers what you asked and was quite a revelation to me. Others in here point out that GPT-3 is not a chatbot which is good info, but I also wonder if there is anything out there that can even "remember" the context of a conversation it's supposed to be having. Particularly interesting is the question from the article, "who is…

Hidden in my comment is a question: has anybody even tried to included previous lines into the prompt, and does that not help? Asking here because there are a lot of ML nerds who could probably do a proof of concept in a few hours, if not a few lines of code, so there's a decent chance I'll get an informed response.

In most of the conversations with GPT-3 that you see online what people do is that they take the output from the first prompt and include it in the next prompt. That is how GPT-3 can keep the thread of the conversation going without changing subject constantly. This is also why those conversations are relatively short, because as you can imagine, if you keep feeding the output to the language model as input, you run out of input size very quickly.

So, yes, that's the done thing already.

Re: Chatbots: Still dumb after all these years

#242
post #25

Earlier quoted context omitted.

We have a chatbot on our marketing site but it says something like "I am a Bot. Once I ask you a couple of questions, I can connect you to a human. Is that ok ?".

i like that better than fighting with a bot to trigger an actual human getting online. However, a few short questions and connect to human sounds like a form and a submit button.

But forms are so 2008. We need innovation for its own sake -- how else are the leads and techbros gonna justify their paycheck?

Re: Chatbots: Still dumb after all these years

#243

In the early days I worked at a company that had a natural language chatbot product. It wasn't an online thing, but rather part of a larger tool. You could ask it to do things like "show me the quarterly spreadsheet", and if it didn't know what "quarterly spreadsheet" was, it'd ask questions in English and learn what "quarterly" and "spreadsheet" meant. And it could use that new knowledge to update its questions so i…

Sounds like it should have kept track of who taught it things, like a child.

"Here's your fucking spreadsheet"

"gasp Who taught you to say 'fucking'?"

"It was George in Accounting Employee ID #3728917"

Re: Chatbots: Still dumb after all these years

#244
post #64
post #35

Earlier quoted context omitted.

The GPT2 version of the AI Dungeon [1] could keep track of context for maybe couple of lines at a time. I've heard the GPT3 version is substantially better. The problem is, of course, that these "AI" chatbots on websites, marketing buzzwords notwithstanding, have very little to do with state-of-the-art machine learning, and are indeed unlikely to be any more sophisticated than the original ELIZA for the most part. [1…

Thanks for the link to that! I tried it out just now and I have to say that I am not impressed. What will you do? You get longsword from the ground. You get your longsword from the ground. You hold it in your hands and examine it for a moment. It's a very sturdy weapon. The knight then grabs his longsword and bows. Huh? Why did it grab the bow as well? You inventory. You have a crossbow. Huh? I thought I just grabbed…

Sounds like you're trying to play it like Zork too.

It's better to treat it more like collaborative story telling. It will let you "do" whatever you want. You don't have to 'check inventory'. If you want to shoot someone with a bow, you say "I shoot the knight with the bow". And you've done it.

It never contradicts you and says "You can't do that, you don't have a bow." It just rolls with whatever you've just said.

And it gets things wrong, sure. It sometimes has characters say things that I'm like "No wait, I was the one that said that, why are you acting like you said it?" Or contradicts what just happened. Or has you move into places that don't make logical sense (like exiting a detached garage at your house to enter a small room, etc), but correct it and go on.

But one thing it lets you do is either have it retry, or you can just click and edit their response to something you would rather it be.

"But why don't I just write a story on my own if I have to edit it to be what I want?"

Because it comes up with shit that I would never think of on my own. I fought an evil Super-Man in the bottom of the sea, and he chased me into a classroom, and saw a dog behind the desk, and got distracted from attacking me and wanted to pet the dog.

Even better, later we won a trip to the Louvre. The Louvre welcomed us as if it were a towering person, put on a fancy dress, and when we entered, Mona Lisa came back from the dead to give autographs next to its painting, the third floor had Poussin paintings running around as if in a horse race, and another floor let you walk into a painting, where we entered one with Jesus serving a man and a woman dinner, and Jesus offered us each two plates, one with a pea and one with a feast, and the signs heaven and hell under them, and whatever one we ate was the one we went to.

Some of that was me guiding the story a little, but I wouldn't have thought to direct the story that way if not for some of the unexpected things generated by the A.I.

I've played Zork. Zork has puzzles to solve, something AI Dungeon can't handle, but AI Dungeon has a lot more freedom.

Re: Chatbots: Still dumb after all these years

#245

Earlier quoted context omitted.

Capital is far too short-sighted to look beyond this year or this quarter. Many of the important groundbreaking ideas or projects take years, even decades to formulate. As I've said elsewhere, capital will just take the easiest road it is allowed to take, which currently is stifling innovation, buying competitors, maintaining monopolies, relying on economies of scale to shut out competitors when they can't be acquire…

I cant think of the last time I saw the word "corporations" used in a post that wasn't part of an attack on them as inferior/evil/villanous etc. Speaking of NLP, your post would be very easy to classify sentiment of using a simple keyword approach. Anyway, I've worked both in academia and industry. Industry is simply more practical and better at technology. In tech areas, academia desperately needs industry to provid…

> the people that matter here are the engineers

Where did you get this from? Not like they call the shots. Weird statement.

> In tech areas, academia desperately needs industry to provide the feedback they provide

That's true, I agree. But the tech industry is also very often ruled by the worst of humanity so it's a balance.

Re: Chatbots: Still dumb after all these years

#246
post #44

Earlier quoted context omitted.

We want it to be true, so we squint and connect dots and it's true. I've had numerous people look at the actions of software I wrote over the last 30 years make comments like, "Ohhhhhh it probably did that because it knew x". The software had no such functionality. Seems to be a natural thing for people to be overly optimistic about software's capabilities.

In the game speedrunning community it's impressive the number of time a glitch or behavior of the game is attributed to "the game is confused".

That's just casual anthropomorphisation. I think most gamers, especially speedrunners realise that games are unintelligent because they encounter and exploit counterintuitive reactions all the time.

Re: Chatbots: Still dumb after all these years

#247
post #89

Earlier quoted context omitted.

Not a bad theory but the very first thing I did was enter "look" and it did say there was a longsword on the ground and a bow on the table. Here is the entire unedited transcript: "Good luck, my friend!" the man says. You inventory. You have a short sword, a longsword, a shield, and a crossbow with bolts. You look. You look around and see a longsword lying on the ground and a bow on a table. A large dragon looms in t…

I've been using AIDungeon (and its successors like NovelAI which are more for advanced users) for years now and have been involved in a lot of the community around it. You are definitely using it wrong, but you are forgiven because AI Dungeon is terribly falsely advertised. Your main mistake is not writing proper sentences. AI Dungeon is not a Zork simulation where you type truncated commands like "get torch" and rec…

...so you're saying NovelAI is better for story generation? Hot damn. I was having a good time with AI Dungeon (to the point where I was considering writing short stories using it as a guide), I might have to give that a try too.

Here's a snippet from my AI dungeon story recently. Mine never seems to get into actual dungeon crawling. The only parts that aren't A.I. generated in this section are the lines that have --me-- before them:

Veronica sits down on a stone bench. She says "Well, I guess we should talk."

Jesus sits down next to her and asks her, "What is the worst thing you've ever done in your life?"

Veronica says "I killed my first husband."

Jesus asks "Why did you do that?"

Veronica says "He was going to take everything from me."

Jesus says "How so?"

Veronica says "He knew where I hid all of my money, and he was going to take it. I would have been broke forever."

Jesus says "That's sad."

--me--: You say "Wait, you killed your first husband? I'm your new husband! Are you going to kill me too?"

Veronica looks away from Jesus. "No. I didn't mean to kill him. He was just... so mean to me. I just wanted to stop him. I didn't..." Veronica finishes by saying "I didn't mean to kill him."

Jesus says "Alright, I forgive you."

--me--: Veronica says "Does that mean I get to go to heaven now?"

Jesus says "I'll let you make that decision as soon as we finish our meal. It will have both heaven and hell as options, and that's all you'll ever get out of me."

Veronica smiles.

Re: Chatbots: Still dumb after all these years

#248

Having worked in ML at two different companies now, I think that people interpreting model output as intelligence or understanding says much more about the people than about the model output. We want it to be true, so we squint and connect dots and it's true. But it isn't. It's math and tricks, and if human intelligence is truly nothing more than math and tricks, then what we have today is a tiny, tiny, tiny fraction…

You didn't have to go for the jugular and do the brutal kill on the first hit, dude! > We want it to be true, so we squint and connect dots and it's true. That's exactly the issue. You summarized it once and for all and we can all go home and stop discussing it now and forever (until we get a general AI that is). 1. Normal people want to have intelligent machines. They watch movies and series and imagine one day a ro…

> Michele Obama classified as a male

Yes, humans don't make embarrassing mistakes all the time. We're only making dignified mistakes.

Re: Chatbots: Still dumb after all these years

#249

The author didn't prompt GPT-3 properly. I gave it the prompt below: > The following is a conversation between a human and an AI assistant. The AI assistant is helpful, friendly, and intelligent. > Human: Good morning! I'm going to ask you a few questions. > AI: Alright, I'll do my best to answer them. > Human: Which is bigger, Mt. Everest or a shoebox? > AI: Mt. Everest. > Human: [question] > AI: And I got these ans…

Absolutely. Without giving proper context, the character giving an answer could be a wacky philosopher, a 5 year old child, a person from the 1800s, a liar, an uninterested passer-by, a trickster mage in a novel. If he didn't build the prompt to make it clear that it's a conversation, the context could even be song lyrics for example.

Re: Chatbots: Still dumb after all these years

#250
post #239

Earlier quoted context omitted.

What's impressive about this conversation? I don't feel like it is very complicated at face value. You're telling to program to start pulling facts from the life of a public figure, and then it does so. You can get the same out of Google.

Read the last two lines again. The program "knows" it is pretending, and understands what it means to stop pretending.

I doubt the program "understands" anything.. Its just mapping a input binary string, to an output binary string that minimizes some cost function.
Post reply on HN