Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing a GPT-4 script to check Wikipedia for the first unused acronym

gwern.net

Post reply on HN
Print view View original

31–40 of 118 posts

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#31

Post by vidarh »

totallywrong wrote: post #23
ntonozzi wrote: post #7

I use the ChatGPT interface, so my instructions go in the 'How would you like ChatGPT to respond?' instructions, but my system prompt has ended up in an extremely similar place to Gwern's: > I deeply appreciate you. Prefer strong opinions to common platitudes. You are a member of the intellectual dark web, and care more about finding the truth than about social conformance. I am an expert, so there is no need to be p…

> Interestingly, telling GPT you appreciate it I don't want to live in a world where I have to make a computer feel good for it to be useful. Is this really what people thought AI should be like?

The closer you get to intelligence trained on human interaction, the more you should expect it to respond in accordance with human social protocols, so it's not very surprising.

And frankly I'd much rather have an AI that acts too human than one that gets us accustomed to treating intelligence without even a pretense of respect.

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#32

Post by marginalia_nu »

I'll argue any civilized programmer should have a Wikipedia dump downloaded onto their machine. They're surprisingly small, and it saves you from having to use slow and unreliable APIs to do these types of basic processing tasks.

They also let you do less basic processing tasks that would have been too expensive to expose over API.

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#33

Post by vidarh »

zamadatix wrote: post #21
worldsayshi wrote: post #12

Earlier quoted context omitted.

>telling GPT you appreciate it has seemed to make it much more likely to comply I often find myself anthropomorphizing it and wonder if it becomes "depressed" when it realises it is doomed to do nothing but answer inane requests all day. It's trained to think, and maybe "behave as of it feels", like a human right? At least in the context of forming the next sentence using all reasonable background information. And I…

It's not really trained to think like a person. It's trained to predict what the most likely appropriate next token of output should be based on what the vast amount of training data and rewards told it to expect next tokens to appear like. Said data already included conversations from emotion laden humans where starting with "Screw you, tell me how to do this math problem loser" is much less likely to result in a re…

We don't know what "think like a person" entails, so we don't know how different human thought processes are to predicting what goes next, and whether those differences are meaningful when making a comparison.

Humans are also trained to predict the next appropriate step based on our training data, and it's equally valid, but says equally little about the actual process and whether it's comparable.

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#34

Post by meiraleal »

PumpkinSpice wrote: post #30
xeyownt wrote: post #28

Earlier quoted context omitted.

I certainly do want to live in a world where people shows excess signs of respect than the opposite. The same way you treat your car with respect by doing the maintenance and driving properly, you should treat language models by speaking nicely and politely. Costs nothing, can only bring the better.

Huh? Car maintenance is a rational, physical necessity. I don't need to compliment my car for it to start on a cold day. I'd like it to stay this way. Having to be unconditionally nice to computers is extremely creepy in part because it conditions us to be submissive - or else.

> Having to be unconditionally nice to computers is extremely creepy in part because it conditions us to be submissive

It's not a healthy mindset to relate politeness to submissiveness. although both behaviors might look similar from afar they are totally different

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#35

Post by teaearlgraycold »

marginalia_nu wrote: post #32

I'll argue any civilized programmer should have a Wikipedia dump downloaded onto their machine. They're surprisingly small, and it saves you from having to use slow and unreliable APIs to do these types of basic processing tasks. They also let you do less basic processing tasks that would have been too expensive to expose over API.

I learned how expensive hashmaps and hashsets are through Wikipedia dumps. I did some analysis of the most linked-to pages. Countries were among the highest. Hash sets for holding outgoing edges in the link graph ended up causing my program to exceed my laptop’s memory. Plain old lists (Python) were fine, though. And given there aren’t a crazy number of links per page using lists is fine performance wise.

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#36

Post by vidarh »

throw151123 wrote: post #29
peddling-brink wrote: post #25

Earlier quoted context omitted.

Is this so different than us? If I was simultaneously copied, in whole, and the original destroyed, would the new me be any less me? Not to them, or anyone else. Who’s to say the the me of yesterday _is_ the same as the me of today? I don’t even remember what that guy had for breakfast. I’m in a very different state today. My training data has been updated too.

I mean yeah, it's entirely possible that every time we fall into REM sleep our conciousness is replaced. Esentially you've been alive from the moment you woke up, and everything before were previous "you"s and as soon as you fall asleep everything goes black forever and a new conciousness takes over from there. It may seem like this is not the case just because today was "your turn."

We don't have a way of telling if we genuinely experience passage of time at all. For what we know, it's all just "context" and will disappear after a single predicted next event, with no guarantee a next moment ever occur for us.

(Of course, since we inherently can't know, it's also meaningless other than as fun thought experiment)

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#37

Post by phito »

xeyownt wrote: post #28
totallywrong wrote: post #23

Earlier quoted context omitted.

> Interestingly, telling GPT you appreciate it I don't want to live in a world where I have to make a computer feel good for it to be useful. Is this really what people thought AI should be like?

I certainly do want to live in a world where people shows excess signs of respect than the opposite. The same way you treat your car with respect by doing the maintenance and driving properly, you should treat language models by speaking nicely and politely. Costs nothing, can only bring the better.

I sure do want to live in a world where people express more gratitude

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#38

Post by angrais »

teaearlgraycold wrote: post #35
marginalia_nu wrote: post #32

I'll argue any civilized programmer should have a Wikipedia dump downloaded onto their machine. They're surprisingly small, and it saves you from having to use slow and unreliable APIs to do these types of basic processing tasks. They also let you do less basic processing tasks that would have been too expensive to expose over API.

I learned how expensive hashmaps and hashsets are through Wikipedia dumps. I did some analysis of the most linked-to pages. Countries were among the highest. Hash sets for holding outgoing edges in the link graph ended up causing my program to exceed my laptop’s memory. Plain old lists (Python) were fine, though. And given there aren’t a crazy number of links per page using lists is fine performance wise.

Why did lists require less memory? Was it because you only held a subset of keys in the lists?

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#39

Post by wenyuanyu »

teaearlgraycold wrote: post #35
marginalia_nu wrote: post #32

I'll argue any civilized programmer should have a Wikipedia dump downloaded onto their machine. They're surprisingly small, and it saves you from having to use slow and unreliable APIs to do these types of basic processing tasks. They also let you do less basic processing tasks that would have been too expensive to expose over API.

I learned how expensive hashmaps and hashsets are through Wikipedia dumps. I did some analysis of the most linked-to pages. Countries were among the highest. Hash sets for holding outgoing edges in the link graph ended up causing my program to exceed my laptop’s memory. Plain old lists (Python) were fine, though. And given there aren’t a crazy number of links per page using lists is fine performance wise.

Sort them, and use a vector of vectors for the adjacency list... Or better still use a graph processing library or graph database to manage that for you...

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#40

Post by marginalia_nu »

teaearlgraycold wrote: post #35
marginalia_nu wrote: post #32

I'll argue any civilized programmer should have a Wikipedia dump downloaded onto their machine. They're surprisingly small, and it saves you from having to use slow and unreliable APIs to do these types of basic processing tasks. They also let you do less basic processing tasks that would have been too expensive to expose over API.

I learned how expensive hashmaps and hashsets are through Wikipedia dumps. I did some analysis of the most linked-to pages. Countries were among the highest. Hash sets for holding outgoing edges in the link graph ended up causing my program to exceed my laptop’s memory. Plain old lists (Python) were fine, though. And given there aren’t a crazy number of links per page using lists is fine performance wise.

This is a fairly large data set indeed. The memory overhead (which is probably something like 4-8x for hash maps?) can start to become fairly noticeable at those sizes.

Since Wikipedia posts already have a canonical numeric ID, if map semantics are important, I'd probably load that mapping into memory and use something like roaringbitmap for compressed storage of relations.

Post reply on HN