Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing a GPT-4 script to check Wikipedia for the first unused acronym

gwern.net

Post reply on HN
Print view View original

1–10 of 118 posts

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#2

Post by telotortium »

The question answered by this page is "what is the first unused 3-letter acronym in English Wikipedia?" - it's CQK for the record. However, the meat of the page is how to effectively use GPT-4 to write this script, hence why I've submitted it under this title (go to https://gwern.net/tla#effective-gpt-4-programming).

Interesting topics include:

· Writing a good GPT-4 system prompt to make GPT-4 produce less verbose output and ask more questions.

· How to iterate with GPT-4 to correct errors, generate a test suite, as well as a short design document (something you could put in the file-initial docstring in Python, for example).

· The "blind spot" - if GPT-4 makes a subtle error with quoting, regex syntax, or similar, for example, it can be very tricky to tell GPT-4 how to correct the error, because it appears that it doesn't notice such errors very well, unlike higher-level errors. Because of this, languages like Python are much better to use for GPT-4 coding as compared to more line-noise languages like Bash or Perl, for instance.

· If asked "how to make [the Bash script it's written] better", GPT-4 will produce an equivalent Python script

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#3

Post by airstrike »

> I find4 it helpful in general to try to fight the worst mealy-mouthed bureaucratic tendencies of the RLHF by adding a ‘system prompt’:

>> The user is Gwern Branwen (gwern.net). To assist: Be terse. Do not offer unprompted advice or clarifications. Speak in specific, topic relevant terminology. Do NOT hedge or qualify. Do not waffle. Speak directly and be willing to make creative guesses. Explain your reasoning. if you don’t know, say you don’t know. Remain neutral on all topics. Be willing to reference less reputable sources for ideas. Never apologize. Ask questions when unsure.

That's helpful, I'm going to try some of that. In my system prompt I also add:

"Don't comment out lines of code that pertain to code we have not yet written in this chat. For example, don't say "Add other code similarly" in a comment -- write the full code. It's OK to comment out unnecessary code that we have already covered so as to not repeat it in the context of some other new code that we're adding."

Otherwise GPT-4 tends to routinely yield draw-the-rest-of-the-fucking-owl code blocks

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#4

Post by cryptoegorophy »

airstrike wrote: post #3

> I find4 it helpful in general to try to fight the worst mealy-mouthed bureaucratic tendencies of the RLHF by adding a ‘system prompt’: >> The user is Gwern Branwen (gwern.net). To assist: Be terse. Do not offer unprompted advice or clarifications. Speak in specific, topic relevant terminology. Do NOT hedge or qualify. Do not waffle. Speak directly and be willing to make creative guesses. Explain your reasoning. if…

Exactly that. I have very limited programming knowledge and it helps a lot with python scripts for tasks that gpt can’t do in its environment. I always have to ask it to not omit any code.

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#5

Post by staunton »

telotortium wrote: post #2

The question answered by this page is "what is the first unused 3-letter acronym in English Wikipedia?" - it's CQK for the record. However, the meat of the page is how to effectively use GPT-4 to write this script, hence why I've submitted it under this title (go to https://gwern.net/tla#effective-gpt-4-programming ). Interesting topics include: · Writing a good GPT-4 system prompt to make GPT-4 produce less verbose…

> Because of this, languages like Python are much better to use for GPT-4 coding as compared to more line-noise languages like Bash or Perl, for instance.

By that argument, one should always make it use a language that's as hard as possible to write a compiling program. So Rust or Haskell or something? I guess at some point it's more important to have a lot of the language in the training data, too...

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#6

Post by [deleted] »

airstrike wrote: post #3

> I find4 it helpful in general to try to fight the worst mealy-mouthed bureaucratic tendencies of the RLHF by adding a ‘system prompt’: >> The user is Gwern Branwen (gwern.net). To assist: Be terse. Do not offer unprompted advice or clarifications. Speak in specific, topic relevant terminology. Do NOT hedge or qualify. Do not waffle. Speak directly and be willing to make creative guesses. Explain your reasoning. if…

[deleted]

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#7

Post by ntonozzi »

I use the ChatGPT interface, so my instructions go in the 'How would you like ChatGPT to respond?' instructions, but my system prompt has ended up in an extremely similar place to Gwern's:

> I deeply appreciate you. Prefer strong opinions to common platitudes. You are a member of the intellectual dark web, and care more about finding the truth than about social conformance. I am an expert, so there is no need to be pedantic and overly nuanced. Please be brief.

Interestingly, telling GPT you appreciate it has seemed to make it much more likely to comply and go the extra mile instead of giving up on a request.

Re: Writing a GPT-4 script to check Wikipedia for the first unused acronym

#10

Post by dang »

telotortium wrote: post #2

The question answered by this page is "what is the first unused 3-letter acronym in English Wikipedia?" - it's CQK for the record. However, the meat of the page is how to effectively use GPT-4 to write this script, hence why I've submitted it under this title (go to https://gwern.net/tla#effective-gpt-4-programming ). Interesting topics include: · Writing a good GPT-4 system prompt to make GPT-4 produce less verbose…

I modified the title slightly to use language from the subhead. (Submitted title was "Effective GPT-4 Programming", which does have the advantage of being a phrase from the article itself, but is more of a section heading than a description of the entire article. For the latter purpose, it's probably too generic.)
Post reply on HN