There's a fantastic service called the internet, which helps with things you don't know about. According to Google, 5 feet is roughly 1.5 meters. That means that the ground is frozen solid on average 1.5 meters below surface level in winter. However much thermal capacity your walls have, it won't be long before your attempt at heating or cooling your living space is competing directly with the outside environment. It is less of a concern in summer, since the difference between inside and outside is much smaller, but critical for energy usage in winter.
As for mold and mildew, you only get that with modern insulation if you have a poor envelope- i.e. drafts in the walls due to improper sealing. It's not the sort of thing drier climates have to worry about at all, but in very humid environments such as where I live, preventing condensation inside is important. That means not having walls that are cooler than the dew point of the air getting inside.
Basements here are very prone to getting mold or mildew, because (a) they have solid walls, (b) are kept cool via the earth acting as a heat bank, and (c) will readily absorb moisture through the stone or cement.
Most around here have some sort of sealant paint or active dehumidification running in summer to avoid mildew. If the basement walls didn't have to be load bearing, solid wall construction would absolutely be the last choice for basements because they're definitely the worst for thermal and moisture resistance properties.