Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards a conversational agent that can chat about anything

ai.googleblog.com

Post reply on HN
Print view View original

71–80 of 108 posts

Re: Towards a conversational agent that can chat about anything

#71

Post by Polylactic_acid »

grawprog wrote: post #61
andbberger wrote: post #59

Earlier quoted context omitted.

You definitely need a model of the world, but you probably don't want to be constructing it during speech. What you want is a generative model that you've trained on the simulation data ahead of time so you can quickly make inferences once deployed.

How would that make a conversation more lifelike? It'll still be a limited model of the world, unless somehow you intend to give it the entire breadth of human experience, and will still fall to the same shortcomings as every chatbot, ask it about something it doesn't know and it'll try and fit a response that gives it away as a bot, act confused in a way that gives it away as a bot or just say something nonsensical.…

You don't even have to trick it with slang. Maybe I am behind but so far I have never seen a bot that can consistently or even most of the time understand context. They seem to all seriously struggle with anything more than a random sentence that was determined a probable result of the last message the user sent.

Re: Towards a conversational agent that can chat about anything

#72

Post by Polylactic_acid »

baron_harkonnen wrote: post #30
v7x wrote: post #13

>Also, tackling safety and bias in the models is a key focus area for us, and given the challenges related to this, we are not currently releasing an external research demo. Read: We don't want the internet to turn our program into Tay 2.0 because we are super duper serious. I get why these companies do this. But I can't help but feel like attempting to eliminate "bias" and tackle "safety" are counter productive to t…

I suspect that the real reason is because non-cherry picked answers don't work as well and that if you interact with it you'll be no more impressed that decades of chatbots that aren't all that much more impressive than ELIZA.

I think part of the reason is that chat bots tend to be racist without any intentional effort to make them so. I saw a post last year where someone built the most default language AI setup using the most popular training data and then got it to rank words based on their positivity. Without any intentional effort the AI ranked English names as positive and African/Asian/etc names as neutral or negative words.

Re: Towards a conversational agent that can chat about anything

#73

Post by taneq »

ufmace wrote: post #69
objectofobjects wrote: post #66

Earlier quoted context omitted.

Hypothesis: Meena thinks they're both students enrolled in the same class.

Further conversation would quickly resolve that. Ask it what paper it's talking about. A human-quality response would be an instantly made-up excuse or explanation of what it thought the state was - like "Oh, I thought you were in my class". Or it could be a more typical computer model and display no sign that it even recognizes that the other party thinks there's a misunderstanding.

It seems to me like these massive-data approaches will eventually give responses that look like your first type, but are actually just statistically likely matches based on the dataset.

The question is whether this approach will ever confer the ability to generate new responses, or is just building a big lookup table already encodes a potential response to any input.

Re: Towards a conversational agent that can chat about anything

#74

Post by sriku »

darepublic wrote: post #47
RugnirViking wrote: post #27

From the very first sample conversation they released: - Human: Hi! - Meena: Hey there! What's up? - Human: Same old, Monday blues...what are you up to? - Meena: I'm procrastinating on a paper I'm supposed to be writing. - Human: are you thinking of pulling an all nighter? - Meena: Possibly. What's your paper on? The bot has already mistaken facts about its own crafted persona and the human in less than 2 meaningful…

these nlp models are getting better and better but what we need ofc is for some model of the world to be constructed during the speech. If I tell you that yesterday I accidentally knocked a glass of water off the table and it fell on to soft carpet you could guess that it survived the fall without shattering. What we need is a chatbot that as you talk to it can update a 3d game/physics engine model from your words so…

Judea Pearl has been talking about this gap like forever now. According to him, causal modeling is the missing piece. I do believe it is a serious step up (i mean, does it take a model to look at 350GB of text to have that presented conversation?) .. however I don't know enough to know whether that is enough.

(Reading TheBookOfNow and Causality in parallel)

Re: Towards a conversational agent that can chat about anything

#75

Post by nl »

TACIXAT wrote: post #42

I would really like a bot that could summarize topics for me then drill down into specifics based on questions I had. Even if it was all just from the Wikipedia page. This would be so nice to triage information while driving.

There's active work on this (although in two separate strands of research).

The first is multi-document summarization. Back in 2018 Google published Generating Wikipedia by Summarizing Long Sequences[1] which did a great job of combining multiple pages into a single Wikipedia-like document. Models have strengthened a lot since then so it should be possible to do better, but I haven't seen any work.

There's also a bunch of work on summarization of a single document into a few sentences.

The second piece of research work is Question Answering. There are multiple directions for that, but looking at recent work on the SQUAD 2.0 leaderboard should give you a good idea of what is possible[2].

Basically, combining these systems should work quite well a lot of the time.

[1] https://arxiv.org/abs/1801.10198

[2] https://rajpurkar.github.io/SQuAD-explorer/

Re: Towards a conversational agent that can chat about anything

#78

Post by artfulhippo »

dnautics wrote: post #77

Easy way to test: Human: "take the last letter of orange and name any animal that begins with that letter" Or: Human: "pick any number from 1 to five, multiply it by two, and say any word with that number of letters"

This is a good test of capabilities, but not of humanity. A normal human response to such an order would be "f* off".

Re: Towards a conversational agent that can chat about anything

#79

Post by dnautics »

artfulhippo wrote: post #78
dnautics wrote: post #77

Easy way to test: Human: "take the last letter of orange and name any animal that begins with that letter" Or: Human: "pick any number from 1 to five, multiply it by two, and say any word with that number of letters"

This is a good test of capabilities, but not of humanity. A normal human response to such an order would be "f * off".

would it be better if it were prepended with "let's play a word game?"
Post reply on HN