AI Engineer here.
To start with, I share some of your skepticism about AI and hype (but I love these problems, so I'm happy to take the risk of overhyping to try to solve these challenges). But there is a lot of real work going on in this space. Though most of these are basically the same answers as you'd get in the article.
> it's all about calling a function that takes some text as an input, and getting some text as output.
Real world AI applications involve non-trivial prompts that are often composed of many different components and dynamically changed based on user interaction with the environment. So it's not quite as simple in practice as just calling an API.
> So is the "engineering" part of this finessing the input and massaging the output?
You could make this claim about all software engineering at the end of the day.
If you want to understand whether or not any of the billion companies shipping "AI" products right now are really doing AI, the big term to ask about is "evaluations". It is not trivial to evaluate the performance of LLM output across a broad range of tasks. However if you're not doing this, then you can't possibly know how your efforts are doing. The companies that are slapping "AI" stickers on old products are largely ignoring this issue.
The next challenge is "how do you improve bad outputs?" Prompt engineering is one solution, but there are potentially may other engineering solutions to recovering from a bad state. None of these are trivial.
A rapidly growing part of this space is working with "agents", that is you have multiple LLMs that are capable of interacting with each other. This area is changing rapidly.
Vector databases are also becoming very important of the work as not all LLM/AI work is just throwing around prompts, but often working with embeddings.
> All of a sudden, everyone's an AI Engineer. Where where these experts hiding five years ago?
It's not that mysterious. Everyone I know working in this space right now was either a very engineering focused data scientist in their last role, or an ML engineer working near this space. In either case they're people that have been interested in this space before that have all the skills necessary to change roles.
> Can the same be said of "AI Engineers"?
At least in my circle, everyone doing this work right now has a long history of working in machine learning and quantitative problem solving. Of course that used to be true of ML engineers as well (and I've not far to many MLEs that don't understand gradient descent).