The most important leverage is to just say no if they won’t agree. It doesn’t matter what level or specialization or education or experience you have. The only way to win a negotiation is to just genuinely not want what the other party is offering if it doesn’t meet your criteria, and turn it down.
Obviously, if you are desperate for a job due to other factors, you may be literally unable to turn down a suboptimal offer. It’s unfortunate that companies bank on this, and sometimes even angle their search criteria to specifically find classes of candidates they can exploit with low offers, inadequate workspace, etc. But it’s a fact of life.
When you’re not in that situation and you are free to turn a place down, then you just have to decide what matters to you and ruthlessly stick to it.
To your other points, I think they are not actually big deals. Relative to other costs, building private space on an existing floor is not that bad. Of course, companies are mostly not worried about the cost but are worried about the anger and reaction from existing staff who hate the open plan space they are stuck in.
If a company has a top-down “no office” culture, like “not even our CEO has an office!” then just walk away. Those people are so far gone it’s not worth it. You know from day one that insincere corporate evangelism matters more to them than your actual work. You are more office furniture and less engineer at that point.
It doesn’t matter if this is ~70% of all IT jobs. They are bad jobs. Ok to take them if you’re in a pinch, but why would anyone choose them? They’ll never be forced to reform their habits to recruit adequate staff that way.
Another big option is to ask the company to rent a dedicated office for you from a co-working space. So you’ll be “remote” except working from a single office that they lease for you (or you could lease it yourself and be reimbursed or grossed up for it). This is exceedingly cheap for well-capitalized companies and if they are serious about your productivity and ability to actually collaborate, they’ll do it (as opposed to only being concerned if it superficially looks like people are collaborating because of the open plan layout).
There are plenty of solutions for a motivated company. Existing office build out is not a serious barrier, only cultural unwillingness to be adaptable to people whose jobs require private space to work.