Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-Prompt-Engineer

github.com

Post reply on HN
Print view View original

101–110 of 166 posts

Re: GPT-Prompt-Engineer

#101

Post by namaria »

judge2020 wrote: post #97
namaria wrote: post #33

Earlier quoted context omitted.

> Run a bunch of experiments and see which does better. That's empiricism. Scientific method implies formulating falsifiable theoretical claims, producing meaningful analysis of experimental conditions, publishing peer-reviewed and reproducible results.

That's if you're trying to become a scientist. You can do science even if your experiment doesn't contain any of those elements besides trying multiple things and observing the outcomes.

Sure, the same way I can practice medicine by putting leeches on people. I'm not trying to become a physician so it's alright to call that medicine.

Re: GPT-Prompt-Engineer

#102

Post by flangola7 »

otikik wrote: post #100
donkeyboy wrote: post #67

Earlier quoted context omitted.

There is a paper on arxiv saying that GPT4 correlation with human evaluators on a variety of tasks with strongly positive. I am also uncomfortable with it, but using GPT4 as a grader is not as bad as you think.

Of course there is strong correlation. That is literally what it was designed to do. The problem is that it will simultaneously say that "cow eggs are bigger than chicken eggs", with the same confidence (and in a way that correlates well with human evaluators). https://www.reddit.com/r/Funnymemes/comments/10ohd2n/chatgpt... So when you get an evaluation you are playing the russian roulette - you may get a decent resu…

I just asked and it told me cows are mammals and do not lay eggs. That Reddit post is not even GPT-4 and is 5 months old, which may as well be the 19th century on AI tech timescales.

Re: GPT-Prompt-Engineer

#103

Post by SirMaster »

hospitalJail wrote: post #95
SirMaster wrote: post #93

Earlier quoted context omitted.

So if you have your PE license and are writing software that controls a commercial airplane system or a spacex rocket, you are not an engineer doing software engineering?

Ahh I forgot to put my usual caveat. Assembly and safety critical C can be considered software engineering. Anything with abstraction, no.

I agree that far too many people call themselves software engineers when they really aren't.

But I just mean to say that software engineering itself is definitely still a real thing and there are many people out there that can and should call themselves software engineers.

Re: GPT-Prompt-Engineer

#104

Post by Deestan »

namaria wrote: post #35
Deestan wrote: post #16

Earlier quoted context omitted.

His genius laid in not only seeing which technologies were coming, but predicting what perversions business people would twist it into.

Creating a computer to find "the ultimate answer to life, the universe and everything", getting a cryptic answer and then creating an even bigger and more complicated computer to find the question is a pretty good satire of generative ai based chatbots of exponentially increasing model size.

Don't remember if it made it into the book, but from the radio series my favorite is the scene where they end up in a nightclub filled with dancing mannequins sprayed with sweat in order to convince people it was popular and come in.

Re: GPT-Prompt-Engineer

#105

Post by rTX5CMRXIfFG »

judge2020 wrote: post #97
namaria wrote: post #33

Earlier quoted context omitted.

> Run a bunch of experiments and see which does better. That's empiricism. Scientific method implies formulating falsifiable theoretical claims, producing meaningful analysis of experimental conditions, publishing peer-reviewed and reproducible results.

That's if you're trying to become a scientist. You can do science even if your experiment doesn't contain any of those elements besides trying multiple things and observing the outcomes.

At this point you’re just deliberately making the definition of “science” ambiguous to conveniently support your point.

Re: GPT-Prompt-Engineer

#106

Post by Deestan »

xmcqdpt2 wrote: post #46

I think one should just use GPT to generate the prompts so as to reduce the human input further still, a kind of gpt-gpt-prompt-engineer-engineer.

And a GPT to receive the output, write a summary, and insert it directly into a CEO's trash folder.

Re: GPT-Prompt-Engineer

#107

Post by immibis »

fatso784 wrote: post #65

This tool doesn’t benchmark based on how a model actually responds to the generated prompts. Instead, it trusts GPT4 to rank prompts simply in terms of how well it imagines they will perform head-to-head. Thus, there’s no way to tell if the chosen ‘best prompt’ actually is the best, because there’s no ground truth against actual responses. Why is this so popular, then (more popular than promptfoo, which I think is a…

Because this is the upside if not the peak of the hype bubble. All you have to do is use GPT for a task, and nobody cares whether it actually works, you still get whatever VC funding is left after the interest rate hikes.

Re: GPT-Prompt-Engineer

#108

Post by edgyquant »

hospitalJail wrote: post #79
skwirl wrote: post #59

Earlier quoted context omitted.

I’m guessing you aren’t from the US because in the US there is not much prestige in the title of engineer, but it seems to be a title people in Europe get weirdly hung up on. If anything, software engineering positions are more prestigious than most traditional engineering positions here.

I was a real engineer for a decade before switching to programming. I cannot use the word software engineer, since its nothing like real engineering. Real engineering was def harder, more math intense, and the stakes were sooo much higher. While many software problems can cause you to lose money, engineering problems can cause you to lose time. Yes it sucks your CAD designer had everything on a 0.05 degree angle and…

“Real engineer,” is such an insufferable way to make your point. I guess only civic engineering is “real.” If you don’t like the term software engineer you’d be real mad if you found out about sound stage engineers. Being an engineer isn’t about engineering solutions to business problems; nope it’s having to deal with bureaucratic hurdles that slow you down that makes one an engineer.

Re: GPT-Prompt-Engineer

#110

Post by albrewer »

flangola7 wrote: post #102
otikik wrote: post #100

Earlier quoted context omitted.

Of course there is strong correlation. That is literally what it was designed to do. The problem is that it will simultaneously say that "cow eggs are bigger than chicken eggs", with the same confidence (and in a way that correlates well with human evaluators). https://www.reddit.com/r/Funnymemes/comments/10ohd2n/chatgpt... So when you get an evaluation you are playing the russian roulette - you may get a decent resu…

I just asked and it told me cows are mammals and do not lay eggs. That Reddit post is not even GPT-4 and is 5 months old, which may as well be the 19th century on AI tech timescales.

The post you're replying to is a case in point. This time it's cow eggs; what next?
Post reply on HN