I think it's not about what is the worst, but about what is necessary. Ultimately, the idea is that there are some basic rules without which the society can not perform. The exact rules vary by culture, but in a Western culture they usually include murdering people, violating property rights, causing substantial bodily and emotional harm, defrauding or defaming someone, etc.
Now, one has to admit our current legal system has gone way beyond that and subverted criminal justice system from "absolute necessities" to "things I love to force other people to do because I think they should be doing those things and too stupid to realize it" - and so we now prosecute people for setting wrong kind of plant on fire and inhaling the smoke, or for cutting other person's hair without consent of government employee, or for listening to a musical tune without paying to some corporation that is considering "owning" it, or for using the idea because somebody else thought about similar idea before and asked government employee to give them "ownership" of it. And so on.
This all looks like abuse, but there's still the original idea behind it - the idea that there are some necessities underlying working society, and these necessities must be enforced. Not because each violation causes grave harm - but because multiple violations would snowball into set of consequences that would make our society very hard to live in.
So if we go to the mugging example, let's assume we say "mugging is no problem, the police should not care about it, there's no harm done, especially compared to what Wall Street bankers are doing". Let's ignore even the facts that people are grievously hurt or murdered in muggings. But even beyond that - what happens if we declare free mugging season?
First of all, more muggings - especially from people that are physically strong, imposing and have no moral problems with it. Then, people that are not so strong would try to defend themselves - e.g. by arming themselves. Then, muggers would arm themselves too, because they found the source of cheap income and are not going to give it up. Then you have two outcomes possible - either the non-muggers succeed, individually or collectively, in raising the potential price per mugging to level so high it's not worth it (i.e. if you are going to end up dead in 1/3 of mugging attempts, your mugging career is not going to be long) or you have situation where you're living in a warzone. Which some people learned to survive in, but would you want to get into such situation?
In either case situation did not exactly improve - in the first case, you just create the police by other name, and probably more harsh to muggers than any current one (vigilantes have little use for Miranda rights and competitive trials) , in the second you get dramatic reduction in quality of life.