Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

futurism.com

Post reply on HN
Print view View original

151–160 of 201 posts

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#151

Post by kuschku »

minimaxir wrote: post #10

Since it was not mentioned in the post, here's a direct link to the Reddit comment corpus likely being used: http://files.pushshift.io/reddit/comments/ The full table (up to end of 2015) is available on BigQuery, with separate tables for each month thereafter: https://bigquery.cloud.google.com/table/fh-bigquery:reddit_p... (there is a similar table for comments) And here's a year-old post I wrote on how to use that R…

That’s 260 gigabyte, that’s actually a tiny dataset, you can query and index that on a normal workstation in seconds.

Even training models on that is possible in realistic times on normal systems with that.

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#152

Post by angli »

gr3yh47 wrote: post #63
romaniv wrote: post #45

Earlier quoted context omitted.

Might have been an exclusively German language problem, or maybe they cleaned it up since that article was published. I have a phone with T9 and it works remarkably well. Heck, I would prefer it to Swype. (Of course, it's not useful on smartphones, because they no longer have real keys.)

t9 was AMAZING and i miss it every day. faster and more accurate than swype for sure, especially since inputs for t9 are deterministic. very easy to text even from your pocket etc - something you cant do on a smartphone - notice texting accidents became more common with the widespread adaptation of smartphones ;)

Texting also became more common with the widespread adoption of smartphones - we'd of course see more accidents as a result, but the medium became massively more useful to a lot more people.

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#153

Post by dragonwriter »

Balgair wrote: post #148
dave_sullivan wrote: post #39

Earlier quoted context omitted.

I'm going to make a prediction: Soon after the first chatbot passes a Turing Test, there will be many more to follow, they will get better and better, and the methods will be so interlinked that there will be no way to defend it as proprietary software. The data too will be open source--the public reddit dataset already has tons of value in it. The question then is, "When everyone has access to free chatbots that can…

Woah! I thought they pretty much already have passed it? Remember Ashley Madison? You had ~12 million heterosexual men (that were cheaters, 6 million 'active' users) trying to talk to ~12k heterosexual women (also cheaters, 10k 'active' accounts). It ends up being about a 1:13,000 ratio. Not only that but MANY of these men had paid actual real money to the site in order to do so, and then continued to do so. The only…

There's no reason 6 million non-randomly selected men would be likely to have an IQ distribution similar to the general population. You can't make up for non-random selection with a larger sample size.

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#154

Post by minimaxir »

kuschku wrote: post #151
minimaxir wrote: post #10

Since it was not mentioned in the post, here's a direct link to the Reddit comment corpus likely being used: http://files.pushshift.io/reddit/comments/ The full table (up to end of 2015) is available on BigQuery, with separate tables for each month thereafter: https://bigquery.cloud.google.com/table/fh-bigquery:reddit_p... (there is a similar table for comments) And here's a year-old post I wrote on how to use that R…

That’s 260 gigabyte, that’s actually a tiny dataset, you can query and index that on a normal workstation in seconds. Even training models on that is possible in realistic times on normal systems with that.

[deleted due to wrongness]

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#155

Post by astrodust »

jwtadvice wrote: post #149

Let's not kid ourselves. The technology will be used by PR firms, advertising companies, political campaigns and governments to pretend, at scale, that there is public consensus on certain issues and to drown social media conversation in particular narratives. Anyone have any good defensive technology ideas?

Where there's AI to bombard you with ads, someone will make AI that scrambles your online presence to confuse them.

Social network camouflage.

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#156

Post by astrodust »

minimaxir wrote: post #18
samfisher83 wrote: post #4

Didn't msft do the same thing with twitter and end up with racist bot? I am not sure how this will turn out.

Tay was capable of learning, which is what lead to the abuse.

Same thing happened when Watson started reading Urban Dictionary. The damage was so bad had to roll it back to an older backup.

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#157

Post by chokma »

minimaxir wrote: post #10

Since it was not mentioned in the post, here's a direct link to the Reddit comment corpus likely being used: http://files.pushshift.io/reddit/comments/ The full table (up to end of 2015) is available on BigQuery, with separate tables for each month thereafter: https://bigquery.cloud.google.com/table/fh-bigquery:reddit_p... (there is a similar table for comments) And here's a year-old post I wrote on how to use that R…

If you want to download the set as a torrent (to save pushshift.io some bandwidth cost), you can do so via

  magnet:?xt=urn:btih:UGFLA4QNEXGEFKYYY5ZU37JIHWEEYY5R&dn=reddit_data&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a80&tr=udp%3a%2f%2fopen.demonii.com%3a1337&tr=udp%3a%2f%2ftracker.coppersurfer.tk%3a6969&tr=udp%3a%2f%2ftracker.leechers-paradise.org%3a6969
which I have put together. It contains the data up to April 2016.

If you want to work with this dataset on your workstation, there are some code examples in https://github.com/dewarim/reddit-data-tools

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#158

Post by nthcolumn »

faitswulff wrote: post #78
draugadrotten wrote: post #22

Back in 2007, mobile phones used a system called T9 from Nuance corp which was trained on a word corpus taken from IRC and similar chats. This caused all kinds of issues - the mobile phones would accept offensive words like "naziparking" but reject normal language like "world peace". Using reddit may lead to ... surprises. Source: http://spraktidningen.se/artiklar/2007/11/darfor-ar-din-mobi... Translated by Google: h…

T9 would autocomplete my name to "Asian Lung," which was hilarious to my high school friends. And to be fair, I am Asian and I do have lungs.

To be fair, most people are Asian and have lungs.

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#159

Post by chatmasta »

tachyonbeam wrote: post #73
chatmasta wrote: post #71

Earlier quoted context omitted.

Why do you not allow google to index discussion threads? Sometimes I can remember the comments on an article, but not the article itself. Unfortunately I can't search google using comment text, because Reddit doesn't allow google to index its comments.

What are you talking about? I search reddit using google all the time !?

https://www.reddit.com/robots.txt

    Disallow: /*/comments/*?*sort=
    Disallow: /r/*/comments/*/*/c*
    Disallow: /comments/*/*/c*

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#160

Post by jwtadvice »

astrodust wrote: post #155
jwtadvice wrote: post #149

Let's not kid ourselves. The technology will be used by PR firms, advertising companies, political campaigns and governments to pretend, at scale, that there is public consensus on certain issues and to drown social media conversation in particular narratives. Anyone have any good defensive technology ideas?

Where there's AI to bombard you with ads, someone will make AI that scrambles your online presence to confuse them. Social network camouflage.

That's a cool idea.

Does anyone know of any projects in this direction?

Post reply on HN