Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Releases Largest GPT-2 Text Generation Model

openai.com

Post reply on HN
Print view View original

101–110 of 166 posts

Re: OpenAI Releases Largest GPT-2 Text Generation Model

#101

Post by blurbleblurble »

TTPrograms wrote: post #38
490d0aff0ee8 wrote: post #26

Tangent rant. I'm skimming over some of the code at https://github.com/openai/gpt-2/blob/master/src/model.py and I can't help but feel frustrated at how unreadable this stuff is. 1. Why is it acceptable to have single-letter variable names everywhere? 2. There's little to almost no documentation in the code itself. It's unclear what the parameters of any given function mean. 3. There are magic constants everywhere. 4…

It's a specification of essentially a complex graph of mathematical operations. If there's a function called def mult(a,b): return a*b it's not much more informative to write: def mult(activation_a, activation_b): return activation_a*activation_b Many of these functions are not much more complex than that, and the names along with their comments are more than sufficient given familiarity with the literature. If you t…

When math is involved, it's much easier to read code with short variable and function names.

Re: OpenAI Releases Largest GPT-2 Text Generation Model

#102

Post by alain94040 »

taneq wrote: post #79
chaz6 wrote: post #8

Surely we are not far off models capable of submission-quality essays that will enable a new generation of cheating.

I'd say generating an entire coherent essay like this would constitute a solid pass of the Turing test. I don't think we're that close to that yet (or alternately the singularity is right around the corner, because you could feed that same bot a corpus of AI papers and have it write new publishable ones.)

But it should be possible to separately train a neural network on basic knowledge, so it can generate a stream of ideas. Then use GPT-2 to generate a sentence or paragraph, following those guidelines.

Just like we humans: we first think of what we want to say (a few points about X, first Y, don't forget Z). Then turn those rough ideas into fully-formed sentences.

Re: OpenAI Releases Largest GPT-2 Text Generation Model

#103

Post by htmk »

FillardMillmore wrote: post #34
SequoiaHope wrote: post #12

Earlier quoted context omitted.

wow I’d love to read the Marxist and anarchist texts it has produced. I wonder if they used good source material. I would bet it gets the talking points but can’t convey the subtlety.

Imagine the hilarity of a robot preaching anarchism, it would make for quite a laugh: "AI good, government bad. Government taxes you and makes you pay. AI thinks for you and does what you say. Imagine the day, AI comes to stay, we'll build the roads for you to play, and make the evil government go away, let's rise together, perhaps today?"

I'd battle against the government with this AI.

Re: OpenAI Releases Largest GPT-2 Text Generation Model

#105

Post by parksy »

Rebelgecko wrote: post #72
vitorbaptistaa wrote: post #45

Earlier quoted context omitted.

Pretty impressive! This is what I got (my initial text is in italic): One day I woke up suddenly after a bad dream. I was sitting on my bed alone. As soon as I turned over I found myself lying down in a strange bed. It had no bed, no mattress and no cover for me. I was just lying on that thick carpet. I looked around to discover nothing but a white bed. There was no window, no furniture. No door. The only clue was a…

I've always wondered if I could spend a weekend with some neural nets, do some light editing, and try to corner the market on some of the more obscure Amazon ebook categories (e.g. Sasquatch Erotica) It seems like this model would be more than adequate. Other than some basic human anatomy mistakes (e.g. breasts are typically above the waist), it generates some surprisingly explicit stuff (which I've mostly elided) ev…

_Lucy turned off the light and slipped under the sheets, feeling Frank draw closer_ to her. He lay next to her and she reached up to caress his face softly and she kissed the side of his neck again, feeling him shiver against her. "You look beautiful." Frank looked down at her and smiled. "You look really good in those clothes, though." She kissed his cheek. "I know." "That's okay," he said and he reached out to kiss her, but then he slipped away. "Do you want anything else?" She smiled. "Nothing." He smiled back at her. He was still wearing his clothes from before. He wasn't as naked as he usually was. She reached around to cup his breast and she pressed her face against his. It was just her and him. "Are you ready for me?" Frank grinned. "Ready for what?" "For you," she said and she leaned over and kissed him again, her hands caressing his skin and his breasts, her mouth moving against his to taste his skin.

* His breasts? I mean sure, this is the 21st century I guess. * Her mouth moving against his to taste his skin... * They're both in bed fully clothed with the lights off, and lamenting how they're usually more naked.

I mean obvious inconsistencies aside, this is super fun to play with.

Re: OpenAI Releases Largest GPT-2 Text Generation Model

#106

Post by parksy »

Rebelgecko wrote: post #72
vitorbaptistaa wrote: post #45

Earlier quoted context omitted.

Pretty impressive! This is what I got (my initial text is in italic): One day I woke up suddenly after a bad dream. I was sitting on my bed alone. As soon as I turned over I found myself lying down in a strange bed. It had no bed, no mattress and no cover for me. I was just lying on that thick carpet. I looked around to discover nothing but a white bed. There was no window, no furniture. No door. The only clue was a…

I've always wondered if I could spend a weekend with some neural nets, do some light editing, and try to corner the market on some of the more obscure Amazon ebook categories (e.g. Sasquatch Erotica) It seems like this model would be more than adequate. Other than some basic human anatomy mistakes (e.g. breasts are typically above the waist), it generates some surprisingly explicit stuff (which I've mostly elided) ev…

Just an afterthought, but you could probably cobble something together that asks someone for some basic prompts, like names, locations, etc, then uses a series of templates to generate a chapter of text that's reasonably coherent and tailored to an individual.

Re: OpenAI Releases Largest GPT-2 Text Generation Model

#107

Post by unexaminedlife »

hint23 wrote: post #33

You can try it at: http://textsynth.org

I started with "Donald Trump is". Ended up with this. Looks plagiarized to me :)

---

Donald Trump is a serial liar, and we're going to prove it," the group said. "You can take it to the bank: his word is worthless. His mouth is a hotbed of vulgarity and hate."

The group released a statement on Twitter, saying that it would not stop trying to make Trump accountable for his words.

Trump has also been attacked over allegations that he sexually harassed his former wife Ivana, the billionaire's first wife, during their divorce in the 1990s. Trump vehemently denied he'd ever done such a thing.

Re: OpenAI Releases Largest GPT-2 Text Generation Model

#108

Post by infectoid »

parksy wrote: post #106
Rebelgecko wrote: post #72

Earlier quoted context omitted.

I've always wondered if I could spend a weekend with some neural nets, do some light editing, and try to corner the market on some of the more obscure Amazon ebook categories (e.g. Sasquatch Erotica) It seems like this model would be more than adequate. Other than some basic human anatomy mistakes (e.g. breasts are typically above the waist), it generates some surprisingly explicit stuff (which I've mostly elided) ev…

Just an afterthought, but you could probably cobble something together that asks someone for some basic prompts, like names, locations, etc, then uses a series of templates to generate a chapter of text that's reasonably coherent and tailored to an individual.

So.

1. Feed in all the Game of Throne books

2. Generate final book

3. Profit?

Re: OpenAI Releases Largest GPT-2 Text Generation Model

#109

Post by moron4hire »

Rebelgecko wrote: post #72
vitorbaptistaa wrote: post #45

Earlier quoted context omitted.

Pretty impressive! This is what I got (my initial text is in italic): One day I woke up suddenly after a bad dream. I was sitting on my bed alone. As soon as I turned over I found myself lying down in a strange bed. It had no bed, no mattress and no cover for me. I was just lying on that thick carpet. I looked around to discover nothing but a white bed. There was no window, no furniture. No door. The only clue was a…

I've always wondered if I could spend a weekend with some neural nets, do some light editing, and try to corner the market on some of the more obscure Amazon ebook categories (e.g. Sasquatch Erotica) It seems like this model would be more than adequate. Other than some basic human anatomy mistakes (e.g. breasts are typically above the waist), it generates some surprisingly explicit stuff (which I've mostly elided) ev…

The open secret is that that is how a lot of that fiction is actually written.
Post reply on HN