Your point is valid and you shouldn't be downvoted. Empirically, we know that high density of residential units is correlated with high rent.
Why is this? It ultimately comes back to the Law of Rent. The amount that landlords are able to charge directly corresponds to the produce obtainable at that location over their next best alternative... and that alternative is likewise set that way up until you reach the "margin" which is the best available rent-free location.
Since all land has been homesteaded, there's not much of a ceiling on rents... they tend towards the entire nominal productivity of the site. And the more desirable that site is, the more astronomical the rent.
We want locations to be desirable, so in effect we actually do want rent to be high. It's just important that everyone share in that rising wealth. In other words, we need to create a high demand for labor, by shifting taxes from labor onto idle landlords. Land speculators are like patent trolls atop the surface of the planet.