Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI releases larger GPT-2 model

openai.com

Post reply on HN
Print view View original

11–20 of 87 posts

Re: OpenAI releases larger GPT-2 model

#11

Post by Judgmentality »

jchw wrote: post #5
minimaxir wrote: post #2

Notably, the 345M model (1.5 GB on disk) is big enough that it's pushing the limits of conventional GPUs, and an alternative method of finetuning the model ( https://github.com/nshepperd/gpt-2/commit/47df6da611716b4826... ) has to be used to prevent the GPU from going OOM. I'm working on tools to streamline GPT-2 text generation: I'm currently porting the code above to gpt-2-simple ( https://github.com/minimaxir/gpt-…

>(e.g. Hacker News titles from a retrained 117M model: https://github.com/minimaxir/hacker-news-gpt-2 ) Wow, thats great. “The Bullshit Bubble” “Fuck you, Bootstrap” “We should give up on America” - they’re practically comedy, yet very believable too.

"How to Make a T-Shirt in 10,000 hours" is my favorite.

Re: OpenAI releases larger GPT-2 model

#12

Post by zrm »

jchw wrote: post #5
minimaxir wrote: post #2

Notably, the 345M model (1.5 GB on disk) is big enough that it's pushing the limits of conventional GPUs, and an alternative method of finetuning the model ( https://github.com/nshepperd/gpt-2/commit/47df6da611716b4826... ) has to be used to prevent the GPU from going OOM. I'm working on tools to streamline GPT-2 text generation: I'm currently porting the code above to gpt-2-simple ( https://github.com/minimaxir/gpt-…

>(e.g. Hacker News titles from a retrained 117M model: https://github.com/minimaxir/hacker-news-gpt-2 ) Wow, thats great. “The Bullshit Bubble” “Fuck you, Bootstrap” “We should give up on America” - they’re practically comedy, yet very believable too.

Has anybody tried feeding it comedy to begin with to see what it spits back out?

Re: OpenAI releases larger GPT-2 model

#13

Post by minimaxir »

zrm wrote: post #12
jchw wrote: post #5

Earlier quoted context omitted.

>(e.g. Hacker News titles from a retrained 117M model: https://github.com/minimaxir/hacker-news-gpt-2 ) Wow, thats great. “The Bullshit Bubble” “Fuck you, Bootstrap” “We should give up on America” - they’re practically comedy, yet very believable too.

Has anybody tried feeding it comedy to begin with to see what it spits back out?

There was an attempt on Reddit but it didn't turn out well, likely because there wasn't enough input data: https://www.reddit.com/r/MachineLearning/comments/bgvzdu/d_j...

(I have an idea for a more proper approach)

Re: OpenAI releases larger GPT-2 model

#14

Post by dannyw »

applecrazy wrote: post #7
minimaxir wrote: post #2

Notably, the 345M model (1.5 GB on disk) is big enough that it's pushing the limits of conventional GPUs, and an alternative method of finetuning the model ( https://github.com/nshepperd/gpt-2/commit/47df6da611716b4826... ) has to be used to prevent the GPU from going OOM. I'm working on tools to streamline GPT-2 text generation: I'm currently porting the code above to gpt-2-simple ( https://github.com/minimaxir/gpt-…

One example from your dataset that gave me a laugh: Hiring technical debt (or "unsortable overtime") How do you hire technical debt?? Edit: another one- “In 2009, Africa power creation was switched on for the Google Earth Darth Vader Imperial Warplane Propaganda” I guess Google is diversifying :P

Outsource a project to a contracting agency and you’ve hired technical debt.

Re: OpenAI releases larger GPT-2 model

#15

Post by tyre »

jchw wrote: post #5
minimaxir wrote: post #2

Notably, the 345M model (1.5 GB on disk) is big enough that it's pushing the limits of conventional GPUs, and an alternative method of finetuning the model ( https://github.com/nshepperd/gpt-2/commit/47df6da611716b4826... ) has to be used to prevent the GPU from going OOM. I'm working on tools to streamline GPT-2 text generation: I'm currently porting the code above to gpt-2-simple ( https://github.com/minimaxir/gpt-…

>(e.g. Hacker News titles from a retrained 117M model: https://github.com/minimaxir/hacker-news-gpt-2 ) Wow, thats great. “The Bullshit Bubble” “Fuck you, Bootstrap” “We should give up on America” - they’re practically comedy, yet very believable too.

This is great. A possible hackathon project for writers. Everyone gets a title and has to write the post

Re: OpenAI releases larger GPT-2 model

#16

Post by mkl »

Judgmentality wrote: post #11
jchw wrote: post #5

Earlier quoted context omitted.

>(e.g. Hacker News titles from a retrained 117M model: https://github.com/minimaxir/hacker-news-gpt-2 ) Wow, thats great. “The Bullshit Bubble” “Fuck you, Bootstrap” “We should give up on America” - they’re practically comedy, yet very believable too.

"How to Make a T-Shirt in 10,000 hours" is my favorite.

I had to do a search to make sure that or something analogous wasn't actually real! It seems incredibly on target.

Re: OpenAI releases larger GPT-2 model

#17

Post by mkl »

burtonator wrote: post #9

If you guys want some background check out this blog post: https://openai.com/blog/better-language-models/ It's fascinating to think an AI wrote that story...

That's actually the same page as the main link (which is scrolled down to the update).

Re: OpenAI releases larger GPT-2 model

#18

Post by Felz »

Oh wow, I was literally just making a toy Discord bot for GPT-2. Guess I'll update it with the bigger model.

EDIT: Done! It takes about 3x longer than it did before to generate a response, so if you try it be very patient. Also, I made this in three hours so I wouldn't be surprised if it goes up in flames at some point.

https://github.com/ScottPeterJohnson/gpt2-discord

Re: OpenAI releases larger GPT-2 model

#19

Post by nprateem »

zrm wrote: post #12
jchw wrote: post #5

Earlier quoted context omitted.

>(e.g. Hacker News titles from a retrained 117M model: https://github.com/minimaxir/hacker-news-gpt-2 ) Wow, thats great. “The Bullshit Bubble” “Fuck you, Bootstrap” “We should give up on America” - they’re practically comedy, yet very believable too.

Has anybody tried feeding it comedy to begin with to see what it spits back out?

It'd probably produce a crap sitcom. But then again so does the BBC these days.

Re: OpenAI releases larger GPT-2 model

#20

Post by nprateem »

burtonator wrote: post #10
minimaxir wrote: post #2

Notably, the 345M model (1.5 GB on disk) is big enough that it's pushing the limits of conventional GPUs, and an alternative method of finetuning the model ( https://github.com/nshepperd/gpt-2/commit/47df6da611716b4826... ) has to be used to prevent the GPU from going OOM. I'm working on tools to streamline GPT-2 text generation: I'm currently porting the code above to gpt-2-simple ( https://github.com/minimaxir/gpt-…

This thing really does scare me and fascinate me at the same time. There's so many nefarious things you could do with this... Generate fake news. spam google. etc. One valuable use could be to generate comedy and parody. You could also make it to sabotage others too. You could set it lose on nazi forums and have them argue with bots constantly.

True. Authors at The Onion should be afraid.
Post reply on HN