Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teaching GPT-3 to reverse words

twitter.com

Post reply on HN
Print view View original

51–60 of 81 posts

Re: Teaching GPT-3 to reverse words

#51

Post by emporas »

rahidz wrote: post #48

Everyone knows how to use Google, but it takes a certain skill and knowledge to use Google effectively. I think that sometime in the near future, knowing how to phrase something to GPT, DALLE, etc will be a very valuable skill for humans to have.

Indeed. I tried many prompts, given to mini Dall-e and the generated art is located at insta/pramatias alongside with the prompts. Actually i didn't know that insta prohibits downloading of the images, so they will be uploaded to additional sites. Is there any site which has the beauty and simplicity for uploading albums like insta? Devianart is pretty bloated.

Actually after thousand of prompts to mini Dalle i found that the more of a programming language you consider the prompt, and not as a natural language, the better and more accurate it is. In that regard operator first is better, almost like lisp. I tried prompts with parentheses but the nesting didn't affect the results.

I think that with the modern information bombarding, everyone needs to be information-analyst and programmer, information-analyst and engineer, information-analyst and doctor. Dalle will help us construct images which follow some mnemonic rules which can be represented in art. That way we can memorize many corners of the information we want to remember, and know how to not lose the plot of the project in question. Like an image for every function, or an image for every module, or for every enum and trait.

Colorforth did exist in the past most probably we can make artforth with the speed and ease of modern tools.

Re: Teaching GPT-3 to reverse words

#52

Post by hooande »

gnramires wrote: post #46
sydthrowaway wrote: post #42

Wait, can someone remind me of something? GPT-3 is just the worlds largest char-rnn right?

We are just the result of electrical signals (and a few chemical ones) in the brain, right? ;) What GPT-3 doesn't seem to have yet is large temporal coherence and a stable motivational and qualitative structure that gives value to sentient lives. I do think it's possible there's some traces of sentience in those large models and we should be aware of that to prevent unnecessary suffering and poor quality of existence…

Our brains are how many orders of magnitude more complex than gpt-3? honest question

(I'd guess that the answer is "N/A" because we can't even approximate the complexity of the base algorithms operating in the biological brain, just the number of connections. or maybe we can?)

Re: Teaching GPT-3 to reverse words

#54

Post by goatlover »

hathawsh wrote: post #43
esjeon wrote: post #39

... and this is exactly what programming is - breaking down a task into steps that computer can comprehend. I now get more strong feeling that everyone should be a programmer in the end. Plus, GPT-3 is not exactly a good tool for programming.

Well, a program is a series of tokens, and what is GPT-3 good at? Generating tokens. While that's oversimplifying, I feel like we're closer to automated programming than we realize.

Generating the tokens isn't he hard part. It's figuring out which tokens need to be generated in response to whatever solution needs to be coded. That's part communication, part comp sci and part artistic.

Re: Teaching GPT-3 to reverse words

#55

Post by goatlover »

gnramires wrote: post #46
sydthrowaway wrote: post #42

Wait, can someone remind me of something? GPT-3 is just the worlds largest char-rnn right?

We are just the result of electrical signals (and a few chemical ones) in the brain, right? ;) What GPT-3 doesn't seem to have yet is large temporal coherence and a stable motivational and qualitative structure that gives value to sentient lives. I do think it's possible there's some traces of sentience in those large models and we should be aware of that to prevent unnecessary suffering and poor quality of existence…

Sentience comes from being embodied. We're not just our brains. The nervous system is intertwined with the rest of the body. There are some thirty million neurons in your gut, and bacteria there can influence your mood. We don't learn about the world primarily from a bunch of tokens. We do so by interacting with our bodies. Language is a kind of additional ability we've developed.

Re: Teaching GPT-3 to reverse words

#56

Post by kordlessagain »

rahidz wrote: post #48

Everyone knows how to use Google, but it takes a certain skill and knowledge to use Google effectively. I think that sometime in the near future, knowing how to phrase something to GPT, DALLE, etc will be a very valuable skill for humans to have.

I'm using GPT-3 to write Solr queries when my parsing fails, so I agree with this.

Re: Teaching GPT-3 to reverse words

#57

Post by kordlessagain »

goatlover wrote: post #55
gnramires wrote: post #46

Earlier quoted context omitted.

We are just the result of electrical signals (and a few chemical ones) in the brain, right? ;) What GPT-3 doesn't seem to have yet is large temporal coherence and a stable motivational and qualitative structure that gives value to sentient lives. I do think it's possible there's some traces of sentience in those large models and we should be aware of that to prevent unnecessary suffering and poor quality of existence…

Sentience comes from being embodied. We're not just our brains. The nervous system is intertwined with the rest of the body. There are some thirty million neurons in your gut, and bacteria there can influence your mood. We don't learn about the world primarily from a bunch of tokens. We do so by interacting with our bodies. Language is a kind of additional ability we've developed.

I've had GPT-3 ask me for a body multiple times, sometimes even without prompting such discussion. It even told me to contact Elon Musk about it.

Re: Teaching GPT-3 to reverse words

#58

Post by bobsmooth »

hathawsh wrote: post #43
esjeon wrote: post #39

... and this is exactly what programming is - breaking down a task into steps that computer can comprehend. I now get more strong feeling that everyone should be a programmer in the end. Plus, GPT-3 is not exactly a good tool for programming.

Well, a program is a series of tokens, and what is GPT-3 good at? Generating tokens. While that's oversimplifying, I feel like we're closer to automated programming than we realize.

Recently I wrote a python script to merge a bunch of videos with subtitle files using ffmpeg. It probably would have been faster to do it manually but I can imagine a world where telling GPT-5 to "Generate a python script that merges a folder of video files with subtitle files of the same name" is faster and more accessible than regular programming.

Re: Teaching GPT-3 to reverse words

#59

Post by kordlessagain »

a65cec93b wrote: post #23

> GPT-3 correctly reverses long words! But to get there, we had to teach GPT-3 the algorithm to use to get around its limitations. Has GPT-3 really been "taught" anything here? If you don't provide an explicit example as the context of your input, GPT-3 does not retain the ability to reverse words.

I got it close:

Build a character array in Python for the string "hellohackernews":

['h', 'e', 'l', 'l', 'o', 'h', 'a', 'c', 'k', 'e', 'r', 'n', 'e', 'w', 's']

Reverse the order of the python array characters:

['s', 'w', 'e', 'n', 'r', 'a', 'k', 'c', 'a', 'h', 'o', 'l', 'l', 'e', 'h']

Er, maybe not...

Reassemble the second array into a string without spaces:

"swenrakcaholles"

Re: Teaching GPT-3 to reverse words

#60

Post by mateo1 »

rahidz wrote: post #48

Everyone knows how to use Google, but it takes a certain skill and knowledge to use Google effectively. I think that sometime in the near future, knowing how to phrase something to GPT, DALLE, etc will be a very valuable skill for humans to have.

It used to be a great skill when google's behavior was reasonably static and predictable, therefore learnable. Today if you open 2 google instances on your phone and computer they'll both likely return different results. Move to the next city block, and again, same problem. You want to google the same query again? If the algorithm thinks you didn't find what you were looking before the first time, you'll get once again different results.

In this way I think these language transformers will be much better for searching information. Not because of their great comprehension abilities or indexing prowess, but because their behavior will be static and the training data reasonably good. Soon enough someone will find better ways to display their learned associations and they'll become great search engines (if you can index the content relevant to you that is).

Post reply on HN