As good as you can afford would be fine. The problem is mortgages and the fact that people are buying more than they can afford. They're buying as much as they will be able to afford over the next 30 years. We've developed a system were some people make large amounts of money by creating the fiction that housing should be as expensive as it is.
There's a lot of evidence that shows that for every dollar of loans made available to students, the price of education rises, and that's a system without supply/demand dynamics since admissions aren't (supposed to be) based on a bidding process. It seems only reasonable to conclude that the availability of mortgages has similarly increased the price of housing. It's a hidden cost of our housing market that banks (and a few others) profit by. The banks loan fictional money, created out of thin air thanks to fractional lending requirements, and collect a percentage of that back every year on a house that only costs what it does because of that loan. Without mortgages, people would be forced to save up and the price of housing would be governed more by the cost of actually constructing houses and the amount that people were able to save before buying rather than a percentage of their income over the next 30 years.
Think, for example, if banks decided to change the standard loan time from 30 years to 40 years. What would happen to housing prices in areas with somewhat constrained supply? Homebuyers would still look at monthly payments and buy as much house as they could afford. So housing prices would increase accordingly and instead of mortgaging 30 years of our futures, we'd be mortgaging 40. We'd be buying the same houses and banks would be making more money. Change 30 years to 20 years and you'd get a similar drop in housing prices.
More than any other thing that we spend money on these days, housing prices have been decoupled from their cost of production and banks are largely the ones skimming that difference between what housing should cost and what it actually costs.