OpenAI's GPT-3 may be the biggest thing since Bitcoin

The quality of writing was very high, so I was convinced I was reading something put together by a human with agency... except it didn’t pass my gut-feeling “how IT works”. It made me suspect that either the algorithm (the described one, not the AI responsible) was off, or that I just didn’t understand AI any more. As I know I don’t have up to date AI knowledge, the algorithm appeared more believable. I hiked deep down the uncanny valley with that one.

> I have a confession: I did not write the above article. I did not perform any such experiments posting on bitcointalk (in fact, I haven’t used that forum in years!). But I did it on my own blog! This article was fully written by GPT-3. Where you able to recognize it?

https://ftp.gnu.org/old-gnu/Manuals/emacs/html_node/emacs_47...

>AC.29 Dissociated Press

>M-x dissociated-press is a command for scrambling a file of text either word by word or character by character. Starting from a buffer of straight English, it produces extremely amusing output. The input comes from the current Emacs buffer. Dissociated Press writes its output in a buffer named `Dissociation', and redisplays that buffer after every couple of lines (approximately) so you can read the output as it comes out.

>Dissociated Press asks every so often whether to continue generating output. Answer n to stop it. You can also stop at any time by typing C-g. The dissociation output remains in the `Dissociation' buffer for you to copy elsewhere if you wish.

>Dissociated Press operates by jumping at random from one point in the buffer to another. In order to produce plausible output rather than gibberish, it insists on a certain amount of overlap between the end of one run of consecutive words or characters and the start of the next. That is, if it has just output `president' and then decides to jump to a different point in the file, it might spot the `ent' in `pentagon' and continue from there, producing `presidentagon'.(15) Long sample texts produce the best results.

>A positive argument to M-x dissociated-press tells it to operate character by character, and specifies the number of overlap characters. A negative argument tells it to operate word by word and specifies the number of overlap words. In this mode, whole words are treated as the elements to be permuted, rather than characters. No argument is equivalent to an argument of two. For your againformation, the output goes only into the buffer `Dissociation'. The buffer you start with is not changed.

>Dissociated Press produces nearly the same results as a Markov chain based on a frequency table constructed from the sample text. It is, however, an independent, ignoriginal invention. Dissociated Press techniquitously copies several consecutive characters from the sample between random choices, whereas a Markov chain would choose randomly for each word or character. This makes for more plausible sounding results, and runs faster.

>It is a mustatement that too much use of Dissociated Press can be a developediment to your real work. Sometimes to the point of outragedy. And keep dissociwords out of your documentation, if you want it to be well userenced and properbose. Have fun. Your buggestions are welcome.

The MIT AI Lab's ITS operating system even had a Dissociated Press device which would let you dissociate any file on the system by using the "DP:" device name prefix:

https://github.com/PDP-10/its/blob/4e2ea8e4d851a0ea1f910c300...

My example problem was to get it to apply to Y Combinator. It managed to misspell the organization which isn't something GPT2 normally does.

Here are some examples:

https://0bin.net/paste/RzN1n+cubZwvHDKc#hV5+TXf4eNRRB9ROuXIT...

After the first time it produced "Dear Paul Graham," I stuck it on the end of my prompt as it tended to reliably produce a cover letter (and one for YC) rather than a blank job application form.

Other than throwing lots of computing power at it: I find that it's useful to include some largely irrelevant flavour text in your prefixes that you can stir until you hit on a combination that triggers the results you want (seemingly random things will send the model off in vastly different directions). When it has good initial output but goes off the rails to you can just terminate it at the last good part and continue from there-- a lot of GPT2's worst sins can be correct with fairly low effort that way.

As mentioned, I think these examples are mostly pretty poor. I suspect that if I worked on it I could probably hit on a prompt that caused it to generate better cover letters... That said, there are perhaps a few things I might not have considered without reading them.