So long “prompt engineering,” we hardly knew ya • Hacker News Board

Re: So long “prompt engineering,” we hardly knew ya

#71

Post by BoorishBears » Wed, Jul 05, 2023, 4:34 AM UTC

parpfish wrote: post #62

BoorishBears wrote: post #51

Earlier quoted context omitted.

I feel like people who write stuff like this don't understand the difference between software and software that delivers value. I mean, you think those graphics optimizations we pour millions of dollars into before releasing AAA games will matter when the GTX 8020 outperforms a 4090? Hint: Delivering value for actual people is rarely the result of sitting on your hands and waiting for the next big platform, or even r…

prompt engineering can definitely deliver value for a customer, but i don't think it will deliver value to a developer in terms of building a sustainable career. a different analogy that gets at my original concern: becoming an expert prompt engineer for an particular LLM is like becoming a power user for a piece of proprietary software that isn't getting any more updates.

The first point doesn't follow the second: there will always be a sustainable career in building software that delivers value.

In tech we take it for granted that just because there's some new hotness everyone wants to jump on it day 1. GPT 5 could drop tomorrow and if your tool delivers value using 3.5, it's not going to magically stop delivering value, and in most verticals people will prefer your battle tested 3.5 to some brand new 5.

And if 5 does simplify prompts for your use case and there'll still only be two options:

- making a the same thing as what you made with 3.5 is now trivial... in which case you still have the mindshare and the distribution solved to a degree your newly enabled clones don't.

- making a better version of what you made is now trivial... in which case you can just as trivially improve your version and already have the mindshare and distribution solved.

At the end of the day software developers often struggle to fit software into the larger ecosystem it slots into before it becomes something valuable, and to be GPT has been an amazing case study in the fact.

I think the use of the word engineering did half the damage, and I think people confusing the twitter memes with the interesting attempts at prompt engineering (via ReACT, Gorilla, etc) did the other half, but at the end of the day I think a lot of people will be left kicking themselves when "misguided prompt engineers" end up solving real useful problems in ways they didn't think were possible well before we reach the arbitrary goalposts for foundation models that people keep setting up.

Re: So long “prompt engineering,” we hardly knew ya

#72

Post by bsaul » Wed, Jul 05, 2023, 7:14 AM UTC

GenerocUsername wrote: post #37

bsaul wrote: post #25

Am i the only one that tried using gpt4 for serious development work but stopped pretty fast because of its obvious limitations ? Reading that article makes me wonder if we're even talking about the same thing.

You are not the only one, but also your experience is not the only experience with the tool. I have used several of 'the big' SaaS GPTs, and have gotten great use from them. There is absolutely a use for these tools, but they do take some small amount of skill to get good results. It all comes down to the context you can provide to steer the answer to what you really need. The better you can describe your current sta…

When working on a project, context is everything. I feel like it'll only really make a difference once it's able to read the thousands of files in my project's folder.

My only use so far is get inspiration for type naming, and very simple scripts that i'm too lazy to write myself.

But this happens less than once a month.

Re: So long “prompt engineering,” we hardly knew ya

#73

Post by ElFitz » Wed, Jul 05, 2023, 11:23 AM UTC

criddell wrote: post #69

ElFitz wrote: post #65

Earlier quoted context omitted.

That's also very US-centric. In many countries it's more a title associated to education and specific degrees from specific schools than a license to practice some professional organisation could revoke. While, from my limited knowledge, doctors and lawyers both seem to have some sort of license and controlling body in most, if not all, countries.

> That’s also very US-centric. There are quite a few jurisdictions where some Engineer title is protected: https://en.wikipedia.org/wiki/Regulation_and_licensure_in_en...

I haven't counted many this page where there is a regulating body that could take away a license to practice as an engineer.

The UK seems to discriminate on a per discipline basis. Canada apparently is ambiguous, with self-regulating bodies but courts dismissing cases regarding job titles. Germany has one, but only for civil engineers (still according to the wikipedia page).

Still only relying on this wikipedia page, there are on the other hand many countries where although the title is protected, it simply requires one to have studied a certain number of years (Poland), to have completed a specific degree (Brazil, Chile, Germany), or a specific degree in one of a few select higher-education schools (France, Turkey).

Re: So long “prompt engineering,” we hardly knew ya

#74

Post by unaindz » Wed, Jul 05, 2023, 5:45 PM UTC

noogle wrote: post #66

beachy wrote: post #30

Earlier quoted context omitted.

> if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff. Back in the day, when those new fangled relationship database things came on the scene, do you think people would have been well advised to try and find work on the actual database engine itself, instead of the more frivolous work of using the new technology to, say, solve actual business problems?

"prompt engineering" is a self-destructing field. If you use any rigorous approach to optimizing the prompt, you end up with essentially supervised machine -learning: models can (and do) learn the optimal prompt once there is a yardstick for the goodness of the model's response. That's a classical for a data-scientist, but the skill set has little to do with prompts. If you are not rigorous, then what you are doing i…

I don't have much experience with GPT but with image generation.

You need some amount of experimentation to get the best results but in my experience what works for one model does nothing or worsens the output in others. Adding loras and different types of images into the equation makes this so variable that I would never consider it useful besides keeping a few key words I used to get x good result on y model and experimenting with those when I start a new project.

Calling it "prompt engineering" seems odd.

Re: So long “prompt engineering,” we hardly knew ya

#75

Post by kaycey2022 » Thu, Jul 06, 2023, 3:09 AM UTC

JimtheCoder wrote: post #6

parpfish wrote: post #4

i feel like people that get really into prompt engineering seem to forget that other people are working on building models that are even better . Do they really think that the tricks they developed to make gpt3.5 work well will still be necessary on gpt9 (or whatever new model replaces gpt)? if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff.

"if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff." I think this is more for people, though, that want to maximize the use of AI in their own field, isn't it? The "knowledge worker enhancer"? I am not sure they really need to work on the actual AI stuff...

You know I'm something of a "Google User" myself

Re: So long “prompt engineering,” we hardly knew ya

#76

Post by vidarh » Sat, Jul 08, 2023, 10:04 PM UTC

noogle wrote: post #66

beachy wrote: post #30

Earlier quoted context omitted.

> if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff. Back in the day, when those new fangled relationship database things came on the scene, do you think people would have been well advised to try and find work on the actual database engine itself, instead of the more frivolous work of using the new technology to, say, solve actual business problems?

"prompt engineering" is a self-destructing field. If you use any rigorous approach to optimizing the prompt, you end up with essentially supervised machine -learning: models can (and do) learn the optimal prompt once there is a yardstick for the goodness of the model's response. That's a classical for a data-scientist, but the skill set has little to do with prompts. If you are not rigorous, then what you are doing i…

Focusing on specific incantations, yes. Focusing on how to decomposing a problem, probably not, but then you get very close to designing systems / data design and analysis methodologies more than "prompt engineering", so I guess I mostly agree with you in as much as the relevant field is not really about AI as it is about picking up more structured design and analysis practices (again).