The problem is I think you're also disagreeing with yourself in a way.
> That is not our job! Our job is to solve business problems within the constraints we are given. No one cares how well it runs on the hardware we're given. They care if it solves the business problem. Look at Bitcoin, it burns hardware time as a proof of work. That solves a business problem.
Cost is a business problem. It is a constraint. The problem is sometimes it doesn't feel that way because a lot of folks around you use bad practices and it's easy to horizontally scale things so performance is often brushed off.
Now that the era of 0% interest has ended, companies are actually starting to take a look at what they're running and... surprise! Taking even 10% longer to write quality, performant code can yield 2-3x (sometimes I've seen 1000x) improvements which means much lower cost.
> Some programmers work in industries where performance is key but I'd bet not most.
Because most don't know any better. Unfortunately this is a symptom of bootcamps, etc. that have created a lot of "programmers" that only know how to code vs. "engineers" that know how to build systems that are maintainable, scale, and solve complex problems.
In almost every company I've been at they inadvertently retroactively look back and realize how wasteful they were. In the ML world at least, it's a bit different I suppose because performance matters from the start.