Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook uses 1.5B Reddit posts to create chatbot

bbc.com

Post reply on HN
Print view View original

41–50 of 235 posts

Re: Facebook uses 1.5B Reddit posts to create chatbot

#41

Post by msla »

RealityVoid wrote: post #36
msla wrote: post #8

Here's an example of a GPT2 model trained on the /r/Wikipedia subreddit: https://old.reddit.com/r/SubSimulatorGPT2/comments/dghmnm/li... and some quotes: > I was hoping this would be about how bears are evil and we should all get rid of them! I am very disappointed! Also, a conversation: > I love the list. I feel like I should read more. And reply: > The list is a bit long, but the bear is one of my favorite fictiona…

One of my favourite threads on that subreddit were bots debating if they live in a simulation. Pretty funny, in a way. https://www.reddit.com/r/SubSimulatorGPT2/comments/ez6qtj/do...

There's /u/awliasGPT2Bot, a bot trained on /r/AWLIAS, the "Are We Living In a Simulation" subreddit, which comes up with good ones:

https://old.reddit.com/r/SubSimulatorGPT2/comments/caaq82/we...

> We are likely created by a computer program

And:

https://old.reddit.com/r/SubSimulatorGPT2/comments/cbauf3/i_...

> I am an AI

And then there's /u/singularityGPT2Bot, with this one:

https://old.reddit.com/r/SubSimulatorGPT2/comments/c6m6tw/do...

Title:

> Do you think A.I. will be the downfall of humanity or the savior?

And this comment chain:

> The downfall of humanity because of our own naiveté about how the world works.

Reply:

>> The downfall of humanity because of our own naiveté about how the world works.

> How did we get here?

And reply to that:

> Because we were too stupid to realize that we were in a simulation.

Re: Facebook uses 1.5B Reddit posts to create chatbot

#42

Post by canjobear »

sterlind wrote: post #35
M2Ys4U wrote: post #3

> Numerous issues arose during longer conversations. Blender would sometimes respond with offensive language, and at other times it would make up facts altogether. I mean, to be fair, I've had many conversations like that...

That's what happened with Microsoft's Tay. Is it possible to learn sentiment analysis from Reddit? If they had access to modmail they could determine what's offensive to individual subreddits or groups, but I'm not sure if there's a way to gauge fiery reactions without that. Maybe you could bootstrap it with an existing sentiment analysis tool, but that could easily lead to Garbage In Garbage Out.

Tay went bad because of a different mechanism. In the case of Tay, trolls figured out a way to make it repeat back arbitrary strings, and used that to create seemingly offensive dialogue. In the case of this chatbot, the offensiveness is coming from the underlying training data.

Re: Facebook uses 1.5B Reddit posts to create chatbot

#43

Post by sailfast »

minimaxir wrote: post #22

A reminder that you can obtain the majority of Reddit posts/comments via BigQuery (via Pushshift). No need to write your own scraper. https://console.cloud.google.com/bigquery?p=fh-bigquery&d=re... https://console.cloud.google.com/bigquery?p=fh-bigquery&d=re... It appears to be roughly up to August 2019 for posts, October 2019 for comments.

That's interesting!

Did Facebook ask permission to create derivative works (the bot) from Reddit posts, I wonder, or does this fall under web-scraping law?

If I recall Reddit users still retain rights to their posts unless Reddit the company provides some sort off broad grants?

If they did not, this is an interesting example a company potentially making a great deal of money (if the bot is sold as something) from content that legally belongs to users without compensation. It's one thing if it abides by a site user agreement and users understand once they post it's gone, but to see it happen from a Reddit corpus seems odd.

Shorter version: source data has value and users should share in any value derived from their data if they have the rights to it.

Re: Facebook uses 1.5B Reddit posts to create chatbot

#46

Post by jonathankoren »

canjobear wrote: post #42
sterlind wrote: post #35

Earlier quoted context omitted.

That's what happened with Microsoft's Tay. Is it possible to learn sentiment analysis from Reddit? If they had access to modmail they could determine what's offensive to individual subreddits or groups, but I'm not sure if there's a way to gauge fiery reactions without that. Maybe you could bootstrap it with an existing sentiment analysis tool, but that could easily lead to Garbage In Garbage Out.

Tay went bad because of a different mechanism. In the case of Tay, trolls figured out a way to make it repeat back arbitrary strings, and used that to create seemingly offensive dialogue. In the case of this chatbot, the offensiveness is coming from the underlying training data.

It was a bit of both.

There was a post after Tay came out that argued that Tay's answer to "Is Ted Cruz the Zodiac Killer?" came from the training the data, because that was already a meme, and it came back with the quip within minutes of launch.

Re: Facebook uses 1.5B Reddit posts to create chatbot

#47

Post by floatingatoll »

lowdose wrote: post #2

How can facebook turn a dumpster fire like reddit in a bot that response with more empathy than a human? Didn't Facebook just merge all fb messenger and whatsapp data and trained a NN on the new chat db?

It's not just the 'messages' they're using, it's the 'voting' and 'moderation' human inputs as well.

If a comment is deleted by moderators, the absence of that comment influences the outcomes of using the dataset.

Facebook has no such human moderation of all conversations. Neither does Twitter. That's why it didn't turn quite as evil as the Microsoft bot.

But in the end, this all critically depends on human beings making human judgments and having those taken into account when training the bot. The text itself is secondary. If it was just text, Facebook could have trained using their own dataset. This way, they get all the benefits of volunteer moderators (upvotes, downvotes, moderator-deletes all qualify) without having to pay anyone a single penny for their effort.

Re: Facebook uses 1.5B Reddit posts to create chatbot

#48

Post by adtac »

keanebean86 wrote: post #4
lowdose wrote: post #2

How can facebook turn a dumpster fire like reddit in a bot that response with more empathy than a human? Didn't Facebook just merge all fb messenger and whatsapp data and trained a NN on the new chat db?

You: hello Facebook! Facebook chatbot: [deleted]

[ Removed by reddit in response to a copyright notice. ]

Re: Facebook uses 1.5B Reddit posts to create chatbot

#49

Post by gaogao »

sailfast wrote: post #43
minimaxir wrote: post #22

A reminder that you can obtain the majority of Reddit posts/comments via BigQuery (via Pushshift). No need to write your own scraper. https://console.cloud.google.com/bigquery?p=fh-bigquery&d=re... https://console.cloud.google.com/bigquery?p=fh-bigquery&d=re... It appears to be roughly up to August 2019 for posts, October 2019 for comments.

That's interesting! Did Facebook ask permission to create derivative works (the bot) from Reddit posts, I wonder, or does this fall under web-scraping law? If I recall Reddit users still retain rights to their posts unless Reddit the company provides some sort off broad grants? If they did not, this is an interesting example a company potentially making a great deal of money (if the bot is sold as something) from con…

Legally, https://towardsdatascience.com/the-most-important-supreme-co... gives a good example how transformational machine learning classifiers generally fall under fair use. It does raise a good point that generative machine learning, like this, has not been explored legally yet.

This is still research which will likely provide public good if/when they publish results and methods. Probably, they'll do a different dataset for any commercial work given the profanity problem highlighted in the article.

Re: Facebook uses 1.5B Reddit posts to create chatbot

#50

Post by threatofrain »

lowdose wrote: post #2

How can facebook turn a dumpster fire like reddit in a bot that response with more empathy than a human? Didn't Facebook just merge all fb messenger and whatsapp data and trained a NN on the new chat db?

Reddit a dumpster fire? Some of Reddits technical communities have a way higher signal than HN, like Reddit’s math or rust community. Reddit math is more curious about math than HN is about tech.
Post reply on HN