They all have world-class natural harbors. That was far more important for transportation purposes even 60 years ago (and certainly when these cities were established) than it is today. Population centers formed around places where large quantities of goods and people could be shipped from. And that meant harbors.
Good harbors used to create economic powerhouses and centers of immigration, which in turn created large dense cities. The sort of large dense cities that people naturally chose to build when unregulated but that modern policies prohibit (or at least pose obstacles to) in places that are not yet that dense.
Similarly, NY and Boston are situated on and partly limited by major rivers, which provided access to the interior in a time when travel by boat was much faster than over land. You'll find the same is the case for almost every major non-coastal city in the US and worldwide. That was less of an issue for SF, which grew in the railroad era.
There are of course many obstacles to attracting people to a NEW city. But specific types of geography that happen to typically be limited by water are no longer crucial to making a city dense. Access to ocean shipping is helpful, but it's now viable to import and export over land to and from a harbor town.
Edited to add: Boston isn't really a peninsula. The reason these cities are dense is largely about developing prior to the car. LA could be nearly as dense as Boston if it weren't so car-dependent. If it had become a major population center prior to the advent of the car it would have had to be by necessity.
People built dense because of limits on how far one could realistically travel on foot or by equine-powered means. That limited the size of a city far more effectively in Boston than geography did (this is also reflected to a degree in LA's older downtown).
New York was able to grow beyond downtown and midtown Manhattan largely because of rapid transit (elevated rail later displaced in favor of subways) but even that has limits that kept the city dense. (One of those limits of course being that it was less likely to get built in areas that weren't yet at least somewhat dense.)
Think how long it would take to ride from an outer LA suburb to its downtown in a mule cart. That density-enforcing function doesn't apply to the same degree when you have an automobile.
BUT there are limits on how long people are willing to travel to get places, be it by foot or by car. Even in LA, we're reaching the limits of car-enabled sprawl.