Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-Prompt-Engineer

github.com

Post reply on HN
Print view View original

51–60 of 166 posts

Re: GPT-Prompt-Engineer

#51

Post by fdondi »

arbol wrote: post #9
Kiro wrote: post #6

How are they actually ranked?

It seems like a `ranking_system_prompt` is used to rank the output of other prompts, which is pretty cool! > Your job is to rank the quality of two outputs generated by different prompts. The prompts are used to generate a response for a given task. You will be provided with the task description, the test prompt, and two generations - one for each system prompt. Rank the generations in order of quality. If Generation…

Wait, so we are asking the innkeeper if the wine is good? If the model thinks say that the current year is 2021 it will rate higher a response that does?

There should be at a minimum a way to print which comparisons were made, so that you could double-check if you agree with those.

Also, we could modify the prompt to explain why the decision was made.

Re: GPT-Prompt-Engineer

#53

Post by ljlolel »

Kichererbsen wrote: post #5
PUSH_AX wrote: post #4

It would be cool if, given a handful of test cases, you could send those off to the LLM to generate even more test cases. My first thought when looking over this tool was "Why do I have to do all the work?", the ideal scenario is that I give the high level description and the LLM does the hard work to create the best prompt.

I haven't read the article at all, but what you said just reminded me that I've been using ChatGPT to create Dungeons & Dragons campaigns / adventures / worlds (it's crazy good at that) and one trick i've started doing at the end of a fruitful conversation is to ask it to summarize the discussion so far _in a format to be used as a prompt_. it works more or less.

It can also create compressed versions in unicode that expand to approximate the human readable prompt. Pretty wild.

https://www.reddit.com/r/ChatGPT/comments/12cvx9l/compressio...

Re: GPT-Prompt-Engineer

#54

Post by pavel_lishin »

groestl wrote: post #38
brunoluiz wrote: post #34

Isn’t engineering an exact science while prompt engineering is completely not? Although, even software engineering being an exact science, it is a funny one: most of us don’t get certified as like, let’s say, mechanical engineers do. Would they say we are engineers? So perhaps the “engineer” term got overloaded in recent years?

I am a Software Engineer, I am certified (since in my country, they do so, just as with Mechanical Engineers). Still think it's overloaded though.

Canada?

Re: GPT-Prompt-Engineer

#55

Post by archargelod »

axlee wrote: post #19
arbol wrote: post #9

Earlier quoted context omitted.

It seems like a `ranking_system_prompt` is used to rank the output of other prompts, which is pretty cool! > Your job is to rank the quality of two outputs generated by different prompts. The prompts are used to generate a response for a given task. You will be provided with the task description, the test prompt, and two generations - one for each system prompt. Rank the generations in order of quality. If Generation…

How did they rank the ranking prompt?

They prey extra hard for this one, so the almighty Machine God blesses the results and they won't bring down whole thing. /s

Re: GPT-Prompt-Engineer

#56

Post by bravura »

muzani wrote: post #15
mangecoeur wrote: post #11

Should we really call it "engineering" if its a case of "try random things until one of them works without really knowing why"?

Looks more like science here. Run a bunch of experiments and see which does better. But literally the first sentence of the readme is "Prompt engineering is kind of like alchemy."

This is more engineering than science. For a given task, try a bunch of approaches and keep the best one.

Science would be replicable, ie demonstrate that this particular approach to prompt writing yields better prompts than some baseline prompt writing approach across an array of different problems.

Re: GPT-Prompt-Engineer

#57

Post by Apfel »

Usage query: It looks like this could get expensive quite quickly. The approach is great, but with GPT-4 especially, could be very difficult. Is it worth using with 3.5 as a first pass then switching prompts to GPT4 once you've got the best prompt?

Re: GPT-Prompt-Engineer

#58

Post by xmcqdpt2 »

Makhini wrote: post #48
xmcqdpt2 wrote: post #46

I think one should just use GPT to generate the prompts so as to reduce the human input further still, a kind of gpt-gpt-prompt-engineer-engineer.

I think it's exactly what they do here

Is it? I thought it just ranked the (manual) prompts against each other.

Re: GPT-Prompt-Engineer

#59

Post by skwirl »

mellosouls wrote: post #40
brunoluiz wrote: post #34

Isn’t engineering an exact science while prompt engineering is completely not? Although, even software engineering being an exact science, it is a funny one: most of us don’t get certified as like, let’s say, mechanical engineers do. Would they say we are engineers? So perhaps the “engineer” term got overloaded in recent years?

Software engineer has always been a daft, grandiose term that seems aimed at prestige rather than reality in the majority of cases. If coders can call themselves engineers, no reason why anybody else solving puzzles for a living can't.

I’m guessing you aren’t from the US because in the US there is not much prestige in the title of engineer, but it seems to be a title people in Europe get weirdly hung up on. If anything, software engineering positions are more prestigious than most traditional engineering positions here.

Re: GPT-Prompt-Engineer

#60

Post by School-Cotton »

brunoluiz wrote: post #34

Isn’t engineering an exact science while prompt engineering is completely not? Although, even software engineering being an exact science, it is a funny one: most of us don’t get certified as like, let’s say, mechanical engineers do. Would they say we are engineers? So perhaps the “engineer” term got overloaded in recent years?

“Engineer” originally just meant someone who builds engines (in a broad sense of that word). The formal titles requiring certification, etc. are the more recent development.
Post reply on HN