My fuel economy drops about 20% on my petrol car in cold weather city driving. I don't know what causes the difference, perhaps waiting for the car to warm up and windows to defrost.
EVs upsides and downsides should be considered without it needing to be a comparison to ICE. But, yes, running the heater will use fuel. In this case EVs with "250 miles" range are losing 30%~ of that range for three months of the year and a further 20% over the first three years of their life, making them reliably closer to a 140-mile vehicle. People who are planning round-trip travel or commuting need to know this…
I imagine it is so on some air-cooled cars (VW "Beetle", old 911 etc), but on most cars the cabin is heated off the engine's cooling system and does not require additional fuel, so it technically will "use fuel" but no more than the normal operation of the car.