Live data from Hacker News

Ask HN: Is prompt engineering just snake oil?

news.ycombinator.com

31–40 of 206 posts

Re: Ask HN: Is prompt engineering just snake oil?

#31

Is social engineering just snake oil?

Good question. Is political engineering also engineering?

My question is - for social or political engineering, you can gain definite knowledge (spying, observing, researching, information extraction) and then devise something.

How to do the prompt engineering exactly based on what knowledge? How to formulize and document that knowledge so that it isn't just about intuition and gut feeling but rather a learnable and transferable skill?

Re: Ask HN: Is prompt engineering just snake oil?

#32
I like this framing by Michael Hashimoto, on Prompt Engineering:

https://mitchellh.com/writing/prompt-engineering-vs-blind-pr...

Most of what we see on Twitter or YouTube is Blind Prompting. However, it is possible to apply an engineering mindset to prompting and that is what we should call prompt engineering. Check out the article for a much more detailed framing.

Dair AI also has some nice info and resources ( with academic papers) about prompt engineering.

Re: Ask HN: Is prompt engineering just snake oil?

#33

It's okay; software engineering isn't real either. (In many countries, the word "engineer" is regulated -- you can't call yourself an engineer without professional qualifications and oversight.) https://en.m.wikipedia.org/wiki/Regulation_and_licensure_in_...

An engineer is generally liable for what they sign off on. Software engineering isn't anything like that and all products are filled with legalese to restrict liability.

> all products are filled with legalese to restrict liability.

This doesn't seem to apply in the realm I've been working in. We've had to do the exact opposite thing.

In small, B2B software companies you may find that accepting an uncomfortable amount of liability is usually the only way to get your foot in the door. Certain promises have to be made (very carefully!) or no business would occur at all.

For example, if we told our banking clients that "if your front-line app crashes we might have it fixed in 5-7 days" just to be safe, they likely would have little-to-no interest in utilizing our solution. We have to say things like "We will respond within 1 hour and send a polite apology letter to your CTO every time the server makes a weird noise". And, we have to "engineer" our solution so that we have a minuscule chance of hitting that promised target.

Consider that in a some person who has to own all of the pain that comes along with these decision. Is that person not an "engineer" by some arbitrary definition? Sure. But, for the purposes of "a professional weighing all of the pros/cons and making a decision that they intend to accept all consequences for" - I think it's the same effective practice.

Ultimately, "software engineering" is a very broad space with varying degrees of professionalism. At some ends, you will find things that approximate actual engineering. At others, you will find things that appear more academic or artistic. The context is what defines the title of "engineer" for me.

Re: Ask HN: Is prompt engineering just snake oil?

#34
post #28

It's okay; software engineering isn't real either. (In many countries, the word "engineer" is regulated -- you can't call yourself an engineer without professional qualifications and oversight.) https://en.m.wikipedia.org/wiki/Regulation_and_licensure_in_...

Engineering is a practice. Like medicine, but we’re building machines that run in a virtual space. We architect and build houses in our minds. We send those projects to production to make the business money. That’s engineering.

Eh yes and no. "Traditional" engineering professions, ones where you need to get licensed, are all grounded in physics. Software engineering isn't (ignore the fact that bits running on silicon is physics, it's irrelevant). Software engineering is based on best practices and algorithms that are based on...some guy writing them, saying they're good, and others trusting that they're good. There's a million different ways to make something. Million different opinions. Million different cargo cults. It's why making a FE exam for software would be impossible; there's just too many ways to do something and often very little evidence why it's good.

Re: Ask HN: Is prompt engineering just snake oil?

#35
post #33

Earlier quoted context omitted.

An engineer is generally liable for what they sign off on. Software engineering isn't anything like that and all products are filled with legalese to restrict liability.

> all products are filled with legalese to restrict liability. This doesn't seem to apply in the realm I've been working in. We've had to do the exact opposite thing. In small, B2B software companies you may find that accepting an uncomfortable amount of liability is usually the only way to get your foot in the door. Certain promises have to be made (very carefully!) or no business would occur at all. For example, if…

"B2B software companies you may find that accepting an uncomfortable amount of liability is usually the only way to get your foot in the door"

The company would have to accept it. No legal trouble for the developper.

Re: Ask HN: Is prompt engineering just snake oil?

#36
post #10

Prompt engineering is an 'oil' alright. It's a lubricant to make models work better. The idea of engineering being backed by solid mathematical understanding is a relatively post-calculus idea. For the longest time, we built bridges a certain way, because those are the bridges that stood. We were flying planes long before we understood flight. At the end of the day, engineering is the cycle of "identify a pain point…

"We were flying planes long before we understood flight."

This isn't true. Sir George Cayley elucidated several aerodynamic principles significantly in advance of the Wright Brothers first flights. The Wrights took advantage of this knowledge in designing their propellers and planes. We have since learned a lot more about how flight works, but the guiding principles of lift, drag, and thrust predate the mechanical engineering to realize a heavier than air aircraft.

Re: Ask HN: Is prompt engineering just snake oil?

#37
post #32

I like this framing by Michael Hashimoto, on Prompt Engineering: https://mitchellh.com/writing/prompt-engineering-vs-blind-pr... Most of what we see on Twitter or YouTube is Blind Prompting. However, it is possible to apply an engineering mindset to prompting and that is what we should call prompt engineering. Check out the article for a much more detailed framing. Dair AI also has some nice info and resources ( with…

Prompt testing, especially when for q/a pairs where there are multiple right answers, has been bugging me a lot

The article is reasonable, but also shows a big gap in tooling, as the techniques there feel closer to linting & typing then testing once you do more interesting prompts. They don't check the interesting parts..

Re: Ask HN: Is prompt engineering just snake oil?

#38

It's okay; software engineering isn't real either. (In many countries, the word "engineer" is regulated -- you can't call yourself an engineer without professional qualifications and oversight.) https://en.m.wikipedia.org/wiki/Regulation_and_licensure_in_...

I don't think a country charging fees to bless a word makes a profession more or less real.

It does however give it better control, a fixed threshold in terms of training that applicants should pass, and more oversight.

Those are the important parts, being real or not is irrelevant. Snake-oil selling is also a very real profession.

Re: Ask HN: Is prompt engineering just snake oil?

#39

It's okay; software engineering isn't real either. (In many countries, the word "engineer" is regulated -- you can't call yourself an engineer without professional qualifications and oversight.) https://en.m.wikipedia.org/wiki/Regulation_and_licensure_in_...

I don't think a country charging fees to bless a word makes a profession more or less real.

You should rethink that whenever you drive under a bridge or get on a boat. The reason they collapse or sink so infrequently is largely because engineers are accredited and held responsible.

Re: Ask HN: Is prompt engineering just snake oil?

#40
It's a silly name for a real phenomenon. It's better categorized as a form of rhetoric; instead of persuading people to do what you want, you're persuading LLMs using particular patterns of language. But no one is going to call them "prompt rhetoricians". "Prompt writer" would be better, but "writing" is less of the spirit of the times than engineering.
Post reply on HN