Is learning to ride a unicycle hard? It is if you don't have inherently good balance. Is linguistics hard? It is if you don't have a good associative memory. Is math hard? Is carving a wooden bowl on a lathe hard? Is pro-Hockey hard? Is sculpture hard? Metal working? Philosophy? Music? The percentages of people with specific inherent skill sets will vary widely in society, so 'hard' is best defined as 'requiring a sk…
I think the difference here is that riding a unicycle is a fundamentally bounded problem: you balance, you move your feet, you do a couple of other things. By contrast, programming is effectively unlimited in its potential problem domain.
Philosophy? Music?
These are both hard to do really well, but they have a much greater margin for error and often aren't provably correct or incorrect. When I write an academic paper, if I have a typo on the fourth page, the entire rest of the paper won't catastrophically fail. When I write a program, however. . .
Hard is relative.
Very true! But some fields still, at least to my mind, feel harder than others. One (limited) way you could look at "hard" is in the context of universities: I've seen dropout graphs for various majors. It's not at all unusual for someone to go from electrical engineering to com, or CS to English, but I've yet to hear any English majors say, "English was too hard for me, so I switched to Chemical Engineering."