I mean, it's illegal and you're lying to your employer.. so in the end it's about your morale sense.. it doesn't matter much what other people do or think of it. A somewhat stupid analogy is like saying "I'm killing people in my spare time. I get that it's wrong, but that hasn't stopped me from doing it. Am I the only one that does this?" If you find other people doing it won't make it more legal ;-)
How is it illegal? There might be a civil case (can you sue an employee for laziness?) but I am unaware of a criminal statute on slacking off.
If I hire a freelancer to work on project A and charges me X$/h, but instead work on project B and still charge me for that time, it's illegal. Of course, illegal is a fuzzy term, it really depends of how the contract is crafted and, more importantly, how much one would be willing to take legal action against the freelancer (which will most likely turns out more expensive than just letting it go).
I'm no lawyer, but pretty sure it's not written in standard contracts "Employees can work on whatever he/she wants, get to keep all the IP of it, pending that employee doesn't get caught."