Towards a conversational agent that can chat about anything • Hacker News Board

Re: Towards a conversational agent that can chat about anything

#91

Post by abhs » Wed, Jan 29, 2020, 7:58 AM UTC

rozab wrote: post #68

darepublic wrote: post #47

Earlier quoted context omitted.

these nlp models are getting better and better but what we need ofc is for some model of the world to be constructed during the speech. If I tell you that yesterday I accidentally knocked a glass of water off the table and it fell on to soft carpet you could guess that it survived the fall without shattering. What we need is a chatbot that as you talk to it can update a 3d game/physics engine model from your words so…

GPT-2 showed us that simply having a huge parameter space can be enough for internal consistency. Have you read the unicorn valley article it generated?

The output was cherry picked.

Re: Towards a conversational agent that can chat about anything

#92

Post by james_s_tayler » Wed, Jan 29, 2020, 7:59 AM UTC

allovernow wrote: post #89

In my experience, getting a bot to chat about bullshit is not a particularly useful pursuit in itself. There are far more useful ML paths to explore, like guided, interactive decisionmaking in a much more focused information field. And I think these naturally will lead to authentic conversation. But to truly converse beyond an essentially overfit learned response domain requires real world knowledge(1) and learned re…

You also need opinions too at some point.

Re: Towards a conversational agent that can chat about anything

#93

Post by firefoxd » Wed, Jan 29, 2020, 8:03 AM UTC

> My favorite show is Star Trek.

What does it mean when a chatbot says that? Does it mean that it was fed a bunch of shows and this one resonated (computed)? Or is it a filler until you ask it a question it can help with?

Re: Towards a conversational agent that can chat about anything

#94

Post by dnautics » Wed, Jan 29, 2020, 8:10 AM UTC

colorincorrect wrote: post #85

dnautics wrote: post #79

Earlier quoted context omitted.

would it be better if it were prepended with "let's play a word game?"

maybe, but any regular human being might not want to play your stupid word game which isn't even fun at all asides for the AI researcher.

what's with the hostility? I'm just proving that even a 4 year old would be smarter than an AI agent.

Re: Towards a conversational agent that can chat about anything

#95

Post by icebraining » Wed, Jan 29, 2020, 8:27 AM UTC

colorincorrect wrote: post #86

RugnirViking wrote: post #52

Earlier quoted context omitted.

This might sound unrealistically complicated, and the approach suggested here certainly is, but the idea of simulating a world is a very important concept in our current models of human intelligence, with very small children spending a lot of their time learning basic concepts of cause and effect, properties of objects, materials and physics and so on. It reminds me of something I do in my current occupation, with a…

>very small children spending a lot of their time learning basic concepts of cause and effect, properties of objects, materials and physics and so on. i do not believe that young children explicitly/consciously simulate in their minds to learn cause and effect and other things, though this might be up for debate

Who said anything about "consciously"? It doesn't need to be conscious to affect the conversation.

Re: Towards a conversational agent that can chat about anything

#96

Post by red75prime » Wed, Jan 29, 2020, 8:30 AM UTC

taneq wrote: post #73

ufmace wrote: post #69

Earlier quoted context omitted.

Further conversation would quickly resolve that. Ask it what paper it's talking about. A human-quality response would be an instantly made-up excuse or explanation of what it thought the state was - like "Oh, I thought you were in my class". Or it could be a more typical computer model and display no sign that it even recognizes that the other party thinks there's a misunderstanding.

It seems to me like these massive-data approaches will eventually give responses that look like your first type, but are actually just statistically likely matches based on the dataset. The question is whether this approach will ever confer the ability to generate new responses, or is just building a big lookup table already encodes a potential response to any input.

I prefer to think that we need relevant responses rather than new. And I'm not worried whether it's lookup table, given that lookup table that is required wouldn't fit our universe.

Such models have to make generalizations to fit into mere 2.6 billion parameters as ten word sentence presents a possibility of one billion billion different inputs.

Generalizations are limited, of course. Arithmetic doesn't seem to be generalized, for example, which isn't that surprising given that people need around 4 years to be taught to do it (as opposed to learning it by analyzing random texts), and then they usually need pen and paper in addition to their 100 billion neurons to do non-trivial calculations.

I'm not saying that current network architectures are capable of all generalization humans can do. And we can't teach them like we do with children.

Re: Towards a conversational agent that can chat about anything

#97

Post by mschuster91 » Wed, Jan 29, 2020, 8:42 AM UTC

v7x wrote: post #13

>Also, tackling safety and bias in the models is a key focus area for us, and given the challenges related to this, we are not currently releasing an external research demo. Read: We don't want the internet to turn our program into Tay 2.0 because we are super duper serious. I get why these companies do this. But I can't help but feel like attempting to eliminate "bias" and tackle "safety" are counter productive to t…

> Let the programs "learn" what's objective before trying to constrain it with the subjective.

What is "objective" and what is not is not always something that is obvious and implementable in code. The hot topic of course is politics and religion - feed a bot the speeches of Donald Trump and another bot the speeches of Bernie Sanders, both will not be seen as "objective". Train it with both and the bot will still not be seen as "objective".

Re: Towards a conversational agent that can chat about anything

#98

Post by anon012012 » Wed, Jan 29, 2020, 8:53 AM UTC

Since every comment in this thread seems deriding or downplaying for some reason, let me say that I am impressed by the progress, and that any progress at all is still a very important step, and pushing the envelope. We're getting closer and closer, and I love what I've seen from transformer, and from this!

Re: Towards a conversational agent that can chat about anything

#99

Post by sriku » Wed, Jan 29, 2020, 9:25 AM UTC

rahimnathwani wrote: post #81

sriku wrote: post #74

Earlier quoted context omitted.

Judea Pearl has been talking about this gap like forever now. According to him, causal modeling is the missing piece. I do believe it is a serious step up (i mean, does it take a model to look at 350GB of text to have that presented conversation?) .. however I don't know enough to know whether that is enough. (Reading TheBookOfNow and Causality in parallel)

I think you mean the book of why , not of now

Correct. Thanks and sorry about the typo. Too late to edit it, looks like.

Re: Towards a conversational agent that can chat about anything

#100

Post by mkl » Wed, Jan 29, 2020, 9:55 AM UTC

dnautics wrote: post #94

colorincorrect wrote: post #85

Earlier quoted context omitted.

maybe, but any regular human being might not want to play your stupid word game which isn't even fun at all asides for the AI researcher.

what's with the hostility? I'm just proving that even a 4 year old would be smarter than an AI agent.

Not many four year olds could answer those questions. I think the hostility is people reacting as if they were asked to jump through those verbal hoops in a conversation.