> software engineering really isn't real engineering
Most people who call themselves software engineers have not studied engineering at all, let alone passed an engineering course.
I think there are two broad classes of benefit a real SE can provide.
The first is an understanding of how software can be proven correct, and the limitations of that vis-a-vis the real world - power failures, etc. They can help understand what portion of the software needs what level of performance, understandability, correctness, etc. Your core trade-pricing engine's needs are different than the webserver showing the status pages - both are required but the standards for both are not the same.
Second is an ability (and this is what apprenticing and is for in a real engineering career) to judge and design entire systems. We "devs" often get treated as fungible work units and tasked with various little subsystems. A professional engineer entering such a space would be required to obtain a pretty good understanding of the entirety of the company's systems and very good of what they integrate with, such that they can characterize the cost, risk, etc, of the solution space before even beginning to plan the technical solution. Engineers often tell clients that they've envisioned the wrong solution. They're trained to break down silos and integrate solutions where possible. (Not that they all succeed, but that's in the training.)
> if these LLMs are so intelligent and have so much deeper understanding as their emergent capability - why is prompt engineering needed in the first place?
Because they're language models and people are trying to solve things that aren't pure language problems with them. So you need to map the problem to something it can represent and where a solution can be formed, even if that solution may not use an LLM call. A simple example is taking a mathematical word problem and solving it. Currently the LLMs do not model math well so you'd use the LLM to separate out the clauses of the problem and turn it into variables and write an equation from it and then you'd evaluate that snippet of code for the real answer.
And then, depending on what part of what system this was, you may need to verify the parsing and other parts of the pipeline so you would need to build a chain where the results from one call are handed to other systems, maybe just back to the same LLM, with a bunch of examples, for multiple instances of more detailed checking, and then if those answers aren't the same, to another round where it tries to explain the difference and feeds that explanation into a round where you try to reprompt the initial layer and try again. Token limits often constrain how good the instructions and examples can be so you often have a few levels of prompts, nearly bulletproof ones, and shorter ones which are cheaper and allow other use of the token budget but may have more failure cases.
Building with non-deterministic tools isn't impossible, all ropes are different and yet we have rope bridges, but you have to know how they work and how they fail.
> But who's a prompt engineer? [If] I'm a GPT-3.5 prompt engineer do we need to have a job openings for Alpca or StableML prompt engineers?
No, just like architects learning different materials during their career. But all those LLMs are different and you need to experiment (scientifically) to find their characteristics in your area before building on them.