I grew up in the housing industry as the son of a homebuilder. Not to date myself, but I wrote what was probably one of the first cost accounting spreadsheets on the nascent PC of the day using VisiCalc. The spreadsheet took into account all costs associated with building a starter single family home, including land acquisition, financing, municipal fees, material, and labor costs, I.e. everything one one need to run a successful business as a home builder. I know this industry intimately.
Yes, Stoller is correct it’s harder to be a small (i.e. 10-20 homes per year) home builder than it was 40 years ago. But it’s not due to some conspiracy. It’s because the local zoning has become much more arcane and the red tape so Byzantine that the little guys have been effectively pushed out. For instance, you need a full time staff person to negotiate with your local ZBA (Zoning Board Association) to get permission to get something built. Beyond the direct costs, the delays and uncertainties are killer as they eat into your profits. (The gross margin is slim, somewhere between 10-20% on a home for a small time builder. I know - I wrote the damn spreadsheet that calculated it.) As a consequence of all these regulations, only the big builders can afford to absorb and amortize these costs over tens to hundreds of homes. Of course, this also means the big guys still need a fatter profit to to deal with zoning hurdles, long time frames, uncertainties, and what have you, thus margins need to be higher on average even for them. This is what is going on. Not some monopolistic conspiracy.
I looked Stoller up. Here’s the Wiki:
https://en.wikipedia.org/wiki/Matt_Stoller
His gig is railing against monopolies. The housing industry is just another nail for his one-size fits all monopolistic hammer explanations.
The real culprit here actually is a cartel ironically. But this cartel is made up of homeowners who flex their power through local zoning codes that make it near impossible to build new homes, especially apartments.
TLDR: In other words, the YIMBYs are right: the problem is zoning restrictions on new supply. Reform zoning and small time home builders will be back with a gusto.