The biggest factors end up being about political incentives, and it's not happening in every city and country in the developed world, the majority of them end up with some level of one key dynamic:
People who already own property in an area tend to have relatively high political sway in that area, and have a vested interest in things that limit the new supply of housing. It may be explicitly because they want to keep the value of their property high, or because they want to "keep the character of the area" through things like limits on the number of floors new buildings can have, or open space requirements that require a certain amount of space be left open, or to limit multi-family dwellings.
Ultimately even the property owners are also paying the increased cost of living that comes from these things, but the fact that they're not directly connected and take place in the future (i.e., you don't get an invoice for the next decade of increased rent prices when you vote for a proposal to require that apartments be limited to 4 stories) means that the vast majority of people don't consider them.
People who don't yet own property are either people who already live in the area but who tend to have less sway than property owners (even assuming they know what policies to oppose), or people who are coming from outside the area and have had no prior influence on the politics there.
How effective this dynamic is at stopping new housing construction in a particular area (often because of how its laws are structured, but also because of the policy choices pushed by its local politicians) is one of the primary determinants of how much rents and housing prices in an area will rise.
On one end of the spectrum, you've got the Bay Area's huge number of patchwork municipalities, where interests are highly-entrenched, and where any meaningful reform would have to go through every individual municipality or come from the state level, and in which many of the politicians are themselves specifically anti-housing reform. On the other, you have more centralized city governments like Minneapolis with pro-reform politicians that can handle it at the city level (so there's only one war required to get meaningful reform). Minneapolis rents are actually decreasing because they passed housing reform. Often housing reform will merely stabilize rents (because prices also rise some due to inflation and economic growth increasing demand for land in the area, even if lots of housing is being built). Sometimes it can do enough to actually bring them down.
In a sane world where property owners can't hijack the local government, you'd expect prices to rise some with economic growth and inflation, but significant increases in demand would spur building more housing and increased density. And increased prices due to mild inflation and economic growth are fine -- wages will generally increase to compensate for that (and some of the increase comes from nicer housing being built as the economy grows). Hopefully the housing reform movement will continue bringing us toward that saner world.