[deleted]
GPT-3 Creative Fiction
71–80 of 103 posts
Re: GPT-3 Creative Fiction
#72Fine tuning GPT-3 is one of the biggest challenges, because it's behind an API. The weights aren't available to researchers, so we can't make it do anything it doesn't already do. But, that's fair. It's OpenAI's weights; they can keep them locked up if they want to. What caught my attention, though, is that supposedly OpenAI is working on a way to support fine-tuning. If you think about the logistics of that, it's a…
(I work at OpenAI.) I am finishing up our fine-tuning API this weekend :). If anyone on HN would like to try out the fine-tuning API (or want to build something on top of the base API), send me an email (gdb@openai.com) with your use-case and I can try to accelerate you in our invite queue. PS: We're hiring — if you enjoy building APIs with Python/Go/Kubernetes/Kafka or building front-end interfaces in React, then pl…
Are there any products in the pipeline that you're planning to ship? Asking for prospective candidates.
Re: GPT-3 Creative Fiction
#73It writes uncomfortably coherent and witty fatalist My Little Pony fanfiction: > “I’m Princess Celestia, of Equestria.” the alicorn said, “It’s nice to meet you. And I’m very sorry for your untimely demise.” > “Heh. You’re telling me.” the Professor laughed, shaking his head. “I’m dead. I’m… dead. Do you know what that means?” > “I have an inkling.” Princess Celestia said, wincing. > “It means I don’t have to worry a…
I'm not sure if I'm more impressed by the solid understanding of the concept of death, the coherent dialog with people interacting with each other, or the use of "anypony" instead of anyone.
Re: GPT-3 Creative Fiction
#74It writes uncomfortably coherent and witty fatalist My Little Pony fanfiction: > “I’m Princess Celestia, of Equestria.” the alicorn said, “It’s nice to meet you. And I’m very sorry for your untimely demise.” > “Heh. You’re telling me.” the Professor laughed, shaking his head. “I’m dead. I’m… dead. Do you know what that means?” > “I have an inkling.” Princess Celestia said, wincing. > “It means I don’t have to worry a…
What the hell? How much of a world model does it have, that it can do things like write coherently about death as liberation, and come up with examples of things a living thing might be relieved at not having to bother with anymore? How the hell did a computer write this? Are we sure it's not been overtrained and this is actually a verbatim story from somewhere?
Surely this kind of topic came up in its corpus. I would expect it to have sufficient generalization to "understand" and talk about the topic itself. If all instances of this topic in its training data were replaced with the emotions clowns undergo when throwing ripe tomatoes, it would've written about that.
Re: GPT-3 Creative Fiction
#75Earlier quoted context omitted.
Many ML models are like this (anything used in CV e.g. ResNet, VGG). For example, if you want to classify images as being hot dog or not hot dog (classes that do not exist in ResNet), you can take weights from a pretrained ResNet-50 and finetune the last layer based on a small training set of input images labeled hot dog and not hot dog. This lets you reuse the ResNet's feature detector layers, while plugging in spec…
If I understand correctly, I think you and the other poster are describing transfer learning.
same thing
Re: GPT-3 Creative Fiction
#76First off, gwern, lovely blog. The table of contents is incredibly helpful, especially with those little pop-up previews. I would've loved to have GPT-3 available to me two weeks ago. I was building a personal escape room for my wife as a gift, and used huggingface's GPT-2 website to help write some of the world building content. I'm not a particularly good writer, let alone creative, but wanted a few journal pages/n…
I like providing some context that prompts the model for a list if I think I might need a few candidates. "We discovered a series of journals by several authors in the cabin. The authors' names were:" worked pretty well.
Re: GPT-3 Creative Fiction
#77It writes uncomfortably coherent and witty fatalist My Little Pony fanfiction: > “I’m Princess Celestia, of Equestria.” the alicorn said, “It’s nice to meet you. And I’m very sorry for your untimely demise.” > “Heh. You’re telling me.” the Professor laughed, shaking his head. “I’m dead. I’m… dead. Do you know what that means?” > “I have an inkling.” Princess Celestia said, wincing. > “It means I don’t have to worry a…
Wow. What prompt did you give it to obtain this?
Re: GPT-3 Creative Fiction
#78It writes uncomfortably coherent and witty fatalist My Little Pony fanfiction: > “I’m Princess Celestia, of Equestria.” the alicorn said, “It’s nice to meet you. And I’m very sorry for your untimely demise.” > “Heh. You’re telling me.” the Professor laughed, shaking his head. “I’m dead. I’m… dead. Do you know what that means?” > “I have an inkling.” Princess Celestia said, wincing. > “It means I don’t have to worry a…
On IRC, we loved that sample when I generated it - it's not just a hilariously coherent story which makes you want to know more (what is Starswirl up to? what were Professor Endagerment & Doctor Caballeron fighting over?), some of the lines are both alien & deeply funny: apparently in the afterlife, all numbers are odd, including the number '2'.
Re: GPT-3 Creative Fiction
#79It writes uncomfortably coherent and witty fatalist My Little Pony fanfiction: > “I’m Princess Celestia, of Equestria.” the alicorn said, “It’s nice to meet you. And I’m very sorry for your untimely demise.” > “Heh. You’re telling me.” the Professor laughed, shaking his head. “I’m dead. I’m… dead. Do you know what that means?” > “I have an inkling.” Princess Celestia said, wincing. > “It means I don’t have to worry a…
Wow. What prompt did you give it to obtain this?
See https://www.gwern.net/GPT-3#my-little-pony It was a loosely FiO-themed prompt.
Re: GPT-3 Creative Fiction
#80First off, gwern, lovely blog. The table of contents is incredibly helpful, especially with those little pop-up previews. I would've loved to have GPT-3 available to me two weeks ago. I was building a personal escape room for my wife as a gift, and used huggingface's GPT-2 website to help write some of the world building content. I'm not a particularly good writer, let alone creative, but wanted a few journal pages/n…
FWIW, if I was trying to generate an escape room, I think I would probably try to use AI Dungeon. It seems like a natural fit. You could easily describe the room and edit text as you go to make it a transcript of an escape room. "I pick up the journal to read the author's name" etc.