So long “prompt engineering,” we hardly knew ya

I mean, you think those graphics optimizations we pour millions of dollars into before releasing AAA games will matter when the GTX 8020 outperforms a 4090?

Hint: Delivering value for actual people is rarely the result of sitting on your hands and waiting for the next big platform, or even rolling up your sleeves and trying to learn how to build the next big thing.

You have this idea that by investing energy in something that will be obsoleted you're losing out, but spoiler, that's how 99% of software that delivers actual value works.

The cutting edge of tech almost always ends up being PaaS/SaaS serving itself:

Your mail gets to you because someone is working on software with limitations we solved decades ago.

Your paycheck ends up in your bank account because people invest a ton of time in codebases subject to problems we solved long ago.

Your anti-lock brakes aren't built on a Rust codebase, but some horrible memory unsafe mess running on a processor that's a decade out of date.

—

The reality is: 99% of the effort that goes into trying to build the next big thing goes nowhere. The expected value of you trying to learn "the actual AI stuff" to the greater world is near 0 compared to you "just prompt engineering" and putting out something that solves a pain-point nicely with GPT 3.5.

At the end of the day most of the value that gets delivered to actual users comes from engineers who went deeper into extracting value from the current thing.

We still need people to work on the next big thing so that the 1% of effort that isn't wasted can actually materialize... but in my experience the most successful engineers in that regard are still able to realize the delusion it requires, without being paralyzed by the cognitive dissonance that realization invites.

I'm going to link my own snarky titled by actually okay-ish take on prompt engineering, emphasis on the actual engineering, to show that the author doesn't even know what prompt engineering is - https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...

Prompt engineering is not designing a cool prompt. It's when you start applying genuine techniques to do things that are not possible with tokens alone. For example:

"What's the definition of {apple|orange}" where {apple|orange} is the mathematical average of those two words. This is prompt engineering. Right now, Prompt Engineering is basically in Stable Diffusion through Automatic1111, it's in libraries like microsoft Guidance or LMQL, and not a whole lot else.

I think the big problem is actually finding problems you could use LLMs in. I think anyone that's played with them tends to have a good guess at whether they could or couldn't do something (although you need to really test to be sure, do some prompt engineering etc), but actually finding problems to work on that are within the realm of being solvable and useful is the hardest part imo.

1. API, concepts, etc docs with explanation. Much faster then googling then scrolling through tons of texts with blinking adds

2. Write simple things that I don't want to think about. Like in Python process all files in a directory, follow the links. Saves time.

3. Try things I don't know how to do. The recent was checking if user pressed a key without blocking in Python. Nontrivial, but possible. We went through several options till found the one which works on Ubuntu.

So, it's useful, no regrets about subscribing. Funny thing I'm using it working on toy GPT

But, even today prompts and rules of writing are not transferable even between existing LLMs. Future LLMs will have different architectures and requirements. I suspect today's prompt will be split into data and prompt. Probably just references, keywords to actual data.

In other words current state is transitory, next will be very different.