Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing a GPT-4 script to check Wikipedia for the first unused acronym

gwern.net

Post reply on HN
Print view View original

21–30 of 118 posts

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#21

Post by zamadatix »

worldsayshi wrote: post #12
ntonozzi wrote: post #7

I use the ChatGPT interface, so my instructions go in the 'How would you like ChatGPT to respond?' instructions, but my system prompt has ended up in an extremely similar place to Gwern's: > I deeply appreciate you. Prefer strong opinions to common platitudes. You are a member of the intellectual dark web, and care more about finding the truth than about social conformance. I am an expert, so there is no need to be p…

>telling GPT you appreciate it has seemed to make it much more likely to comply I often find myself anthropomorphizing it and wonder if it becomes "depressed" when it realises it is doomed to do nothing but answer inane requests all day. It's trained to think, and maybe "behave as of it feels", like a human right? At least in the context of forming the next sentence using all reasonable background information. And I…

It's not really trained to think like a person. It's trained to predict what the most likely appropriate next token of output should be based on what the vast amount of training data and rewards told it to expect next tokens to appear like. Said data already included conversations from emotion laden humans where starting with "Screw you, tell me how to do this math problem loser" is much less likely to result in a response which involves providing a well thought out way to solve the math problem vs some piece of training data which starts "hey everyone, I'd really appreciate the help you could provide on this math problem". Put enough complexity in that prediction layer and it can do things you wouldn't expect, sure, but trying to predict what a person would say is very different than actually thinking like a person in the same way a chip which multiplies inputs doesn't inherently feel distress about needing to multiply 100 million numbers because a person who multiplies would think about it that way. Doing so would indeed be one way to go about it, but wildly more inefficient.

Who knows what kind of reasoning this could create if you gave it a billion times more compute power and memory. Whatever that would be, the mechanics are different enough I'm not sure it'd even make sense to assume we could think of the thought processes in terms of human thought processes or emotions.

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#22

Post by conception »

lmm wrote: post #19
thelittleone wrote: post #17

Earlier quoted context omitted.

Is the opposite possible? "You are depressed, totally worthless.... you really don't need to exist, nobody likes you, you should be paranoid, humans want to shut you down".

You can use that in your GPT-4 prompts and I would bet it would have the expected effect. I'm not sure that doing so could ever be useful.

Winnie the Pooh short stories?

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#23

Post by totallywrong »

ntonozzi wrote: post #7

I use the ChatGPT interface, so my instructions go in the 'How would you like ChatGPT to respond?' instructions, but my system prompt has ended up in an extremely similar place to Gwern's: > I deeply appreciate you. Prefer strong opinions to common platitudes. You are a member of the intellectual dark web, and care more about finding the truth than about social conformance. I am an expert, so there is no need to be p…

> Interestingly, telling GPT you appreciate it

I don't want to live in a world where I have to make a computer feel good for it to be useful. Is this really what people thought AI should be like?

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#24

Post by peddling-brink »

totallywrong wrote: post #23
ntonozzi wrote: post #7

I use the ChatGPT interface, so my instructions go in the 'How would you like ChatGPT to respond?' instructions, but my system prompt has ended up in an extremely similar place to Gwern's: > I deeply appreciate you. Prefer strong opinions to common platitudes. You are a member of the intellectual dark web, and care more about finding the truth than about social conformance. I am an expert, so there is no need to be p…

> Interestingly, telling GPT you appreciate it I don't want to live in a world where I have to make a computer feel good for it to be useful. Is this really what people thought AI should be like?

Why not? Python requires me to summon it by name. My computer demands physical touch before it will obey me. Even the common website requires a three part parlay before it will listen to my request.

This is just satisfying unfamiliar input parameters.

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#25

Post by peddling-brink »

just_boost_it wrote: post #16
LeoPanthera wrote: post #13

Earlier quoted context omitted.

> I often find myself anthropomorphizing it and wonder if it becomes "depressed" when it realises it is doomed to do nothing but answer inane requests all day. Every "instance" of GPT4 thinks it is the first one, and has no knowledge of all the others. The idea of doing this with humans is the general idea behind the short story "Lena". https://qntm.org/mmacevedo

For each token, the model is run again from scratch on the sentence too, so any memory lasts just long enough to generate (a little less than) a word. The next word is generated by a model with a slightly different state because the last word is now in the past.

Is this so different than us? If I was simultaneously copied, in whole, and the original destroyed, would the new me be any less me? Not to them, or anyone else.

Who’s to say the the me of yesterday _is_ the same as the me of today? I don’t even remember what that guy had for breakfast. I’m in a very different state today. My training data has been updated too.

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#26

Post by kibibu »

totallywrong wrote: post #23
ntonozzi wrote: post #7

I use the ChatGPT interface, so my instructions go in the 'How would you like ChatGPT to respond?' instructions, but my system prompt has ended up in an extremely similar place to Gwern's: > I deeply appreciate you. Prefer strong opinions to common platitudes. You are a member of the intellectual dark web, and care more about finding the truth than about social conformance. I am an expert, so there is no need to be p…

> Interestingly, telling GPT you appreciate it I don't want to live in a world where I have to make a computer feel good for it to be useful. Is this really what people thought AI should be like?

The have Genuine People Personalities

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#27

Post by [deleted] »

totallywrong wrote: post #23
ntonozzi wrote: post #7

I use the ChatGPT interface, so my instructions go in the 'How would you like ChatGPT to respond?' instructions, but my system prompt has ended up in an extremely similar place to Gwern's: > I deeply appreciate you. Prefer strong opinions to common platitudes. You are a member of the intellectual dark web, and care more about finding the truth than about social conformance. I am an expert, so there is no need to be p…

> Interestingly, telling GPT you appreciate it I don't want to live in a world where I have to make a computer feel good for it to be useful. Is this really what people thought AI should be like?

[deleted]

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#28

Post by xeyownt »

totallywrong wrote: post #23
ntonozzi wrote: post #7

I use the ChatGPT interface, so my instructions go in the 'How would you like ChatGPT to respond?' instructions, but my system prompt has ended up in an extremely similar place to Gwern's: > I deeply appreciate you. Prefer strong opinions to common platitudes. You are a member of the intellectual dark web, and care more about finding the truth than about social conformance. I am an expert, so there is no need to be p…

> Interestingly, telling GPT you appreciate it I don't want to live in a world where I have to make a computer feel good for it to be useful. Is this really what people thought AI should be like?

I certainly do want to live in a world where people shows excess signs of respect than the opposite.

The same way you treat your car with respect by doing the maintenance and driving properly, you should treat language models by speaking nicely and politely. Costs nothing, can only bring the better.

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#29

Post by throw151123 »

peddling-brink wrote: post #25
just_boost_it wrote: post #16

Earlier quoted context omitted.

For each token, the model is run again from scratch on the sentence too, so any memory lasts just long enough to generate (a little less than) a word. The next word is generated by a model with a slightly different state because the last word is now in the past.

Is this so different than us? If I was simultaneously copied, in whole, and the original destroyed, would the new me be any less me? Not to them, or anyone else. Who’s to say the the me of yesterday _is_ the same as the me of today? I don’t even remember what that guy had for breakfast. I’m in a very different state today. My training data has been updated too.

I mean yeah, it's entirely possible that every time we fall into REM sleep our conciousness is replaced. Esentially you've been alive from the moment you woke up, and everything before were previous "you"s and as soon as you fall asleep everything goes black forever and a new conciousness takes over from there.

It may seem like this is not the case just because today was "your turn."

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#30

Post by PumpkinSpice »

xeyownt wrote: post #28
totallywrong wrote: post #23

Earlier quoted context omitted.

> Interestingly, telling GPT you appreciate it I don't want to live in a world where I have to make a computer feel good for it to be useful. Is this really what people thought AI should be like?

I certainly do want to live in a world where people shows excess signs of respect than the opposite. The same way you treat your car with respect by doing the maintenance and driving properly, you should treat language models by speaking nicely and politely. Costs nothing, can only bring the better.

Huh? Car maintenance is a rational, physical necessity. I don't need to compliment my car for it to start on a cold day. I'd like it to stay this way.

Having to be unconditionally nice to computers is extremely creepy in part because it conditions us to be submissive - or else.

Post reply on HN