Iceland's hot water was a culture shock to me in 2 ways: 1. The host at our apartment encouraged us to leave the windows cracked and the heat on for good air circulation. 2. The hot water (at the taps) has a sulfer smell, because it's (also) piped geothermal water. My host explained they also had a water heater upstairs in their home because they preferred "heated cold water" over "hot water", which is a funny distin…
This is due to the hot water in those regions literally being pumped out of the ground and into homes, and on a completely separate plumbing system. Majority of other areas use heat exchangers with pristine cold water, thus no smell nor taste is transferred.
So if you are staying in any other municipality in the capitol, you can use the hot water in cooking directly without boiling cold water. It's the same.