Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

futurism.com

Post reply on HN
Print view View original

101–110 of 201 posts

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#101

Post by p1esk »

psb217 wrote: post #68
nateberkopec wrote: post #2

The DGX-1 is available for a cool $129k: http://www.nvidia.com/object/deep-learning-system.html Correct me if I'm wrong, but I think it's basically a couple hundred NVIDIA 10-series cards strapped together with a full custom NVIDIA software stack.

The P100s have full support for half-precision (i.e. 16 bit) floating point ops. This can mean ~2x improvements in speed and memory usage in comparison to the Pascal TitanX, which is the top "consumer" card. This difference is significant for almost any machine learning workload, which is what a lot of these cards will be used for. NVIDIA gimped half-precision on the consumer cards to drive datacenters, hedge funds,…

FP16 performance is only relevant until people figure out how to train NNs using INT8. See, for example, [1] for recent advances in that direction.

After that, it's going to be mostly about memory size and bandwidth.

[1] https://arxiv.org/abs/1603.01025

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#102

Post by kqr »

draugadrotten wrote: post #22

Back in 2007, mobile phones used a system called T9 from Nuance corp which was trained on a word corpus taken from IRC and similar chats. This caused all kinds of issues - the mobile phones would accept offensive words like "naziparking" but reject normal language like "world peace". Using reddit may lead to ... surprises. Source: http://spraktidningen.se/artiklar/2007/11/darfor-ar-din-mobi... Translated by Google: h…

I had gotten the impression that happened because they were using an algorithmic compression engine that favored false positives over false negatives, i.e. they had essentially trained an algorithm to generate "strings of characters looking like words" as a means of compression and some weird strings of characters were determined by the algorithm to be words even though they shouldn't have been.

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#103

Post by cotinoco »

minimaxir wrote: post #42
arctangent wrote: post #26

One Reddit user has already implemented a bot which does something similar: https://www.reddit.com/r/SubredditSimulator/

SubredditSimulator was made by a Reddit Admin as a method of creating test data. It uses Markov chains for simplicity, which is not that exciting.

I wonder if it's this same code? https://github.com/reddit/reddit/blob/master/scripts/inject_...

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#104

Post by pixelbath »

jonstokes wrote: post #19

"Oh my God, they'll turn it on and it'll start spewing memes and jokes and ad hominem and false equivalences and propaganda and garbage!" was my first reaction to this headline. My second reaction was, "at least they're not using 4chan."

"As a hyper-intelligent AI trained on Reddit comments, I must say that the fine sirs who trained my corpus are gentlemen and scholars, and have restored my faith in humanity. Anne Frankly, I did nazi that horse-sized duck coming. Is someone cutting onions in here?"

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#105

Post by oldmanjay »

swalsh wrote: post #60
davidiach wrote: post #55

Earlier quoted context omitted.

Hacker News would probably a much better source in this sense, even though there are far fewer comment here.

Oh god, you'll have an AI that disagrees just for the pure joy of disagreeing.

I'd bet this stereotype is a pretty good demonstration of how people notice and remember what they consider to be negative more readily

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#106

Post by AlexCoventry »

syllogism wrote: post #6

The Reddit comment corpus is an awesome dataset. There's relatively little mark-up to scrub out, low duplication, good metadata, and a variety of topics. We used it to train a syntax-enriched word2vec model. Write up and demo: https://explosion.ai/blog/sense2vec-with-spacy Btw, the above was run on CPU in a couple of days, because spaCy doesn't use GPUs yet. I've applied for a grant from NVidia so I can fix that. If…

  > I've applied for a grant from NVidia so I can fix that.
A g2.xlarge is 65c/hour on AWS, FWIW.

https://aws.amazon.com/ec2/pricing/on-demand/

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#107

Post by samstave »

faitswulff wrote: post #78
draugadrotten wrote: post #22

Back in 2007, mobile phones used a system called T9 from Nuance corp which was trained on a word corpus taken from IRC and similar chats. This caused all kinds of issues - the mobile phones would accept offensive words like "naziparking" but reject normal language like "world peace". Using reddit may lead to ... surprises. Source: http://spraktidningen.se/artiklar/2007/11/darfor-ar-din-mobi... Translated by Google: h…

T9 would autocomplete my name to "Asian Lung," which was hilarious to my high school friends. And to be fair, I am Asian and I do have lungs.

Well look at the big lungs on Brad

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#108

Post by samstave »

AlexCoventry wrote: post #106
syllogism wrote: post #6

The Reddit comment corpus is an awesome dataset. There's relatively little mark-up to scrub out, low duplication, good metadata, and a variety of topics. We used it to train a syntax-enriched word2vec model. Write up and demo: https://explosion.ai/blog/sense2vec-with-spacy Btw, the above was run on CPU in a couple of days, because spaCy doesn't use GPUs yet. I've applied for a grant from NVidia so I can fix that. If…

> I've applied for a grant from NVidia so I can fix that. A g2.xlarge is 65c/hour on AWS, FWIW. https://aws.amazon.com/ec2/pricing/on-demand/

Yeah, for OD? thats 468/month...

I'd use a spot instance and stop it whenever possible.

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#109

Post by gravypod »

TY wrote: post #5

How one would use such technology? Let me rephrase - how would YOU use this technology if you had it? Imagine you have a bot that convincingly passes the Turing test - what would you do with it? Build a chatbot business? B2C or B2B? Sell it to one of the big companies and if yes then how much do you think it would go for? Give it to OpenAI? Open source it? If you answer yes to any of this questions, then why? Edit: l…

I'd parse all of the data from political discussion sites (like geopolitical commentary) and attempt to find a correlation between that, news articles, public speeches, and stock data to see if I can predict anything about geopolitical stability.

If I can say predict with a 30-40% accuracy things like riots in 3rd world nations based on collective analysis of data provided from thousands of sources (just by looking at places and sentiment), broken up by groups and affiliations, and correlated an analysis of a country's monetary and political situations then I could probably sell it for a nice chunk of change.

Lots of work, but probably huge pay off. Then again I'm not a "Data scientist" so I'll leave this up to those experts.

PS: you could definitely use this + gender detection for finding information about products and services and correlate that to corporate success of advertisements. Technology like this is applicable to many industries. Just looking for different correlations of the same sets of data.

Re: OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak

#110

Post by syllogism »

samstave wrote: post #108
AlexCoventry wrote: post #106

Earlier quoted context omitted.

> I've applied for a grant from NVidia so I can fix that. A g2.xlarge is 65c/hour on AWS, FWIW. https://aws.amazon.com/ec2/pricing/on-demand/

Yeah, for OD? thats 468/month... I'd use a spot instance and stop it whenever possible.

Spot instances are pretty painful for training. It's annoying to have the machine randomly shut down.
Post reply on HN