"The thing that worries me about this push towards encryption and anonymity is that it does afford nefarious elements a lot of safe harbor"
I have yet to see any evidence that this is true; all I see are anecdotes from law enforcement agencies who are pushing for less crypto, and even those anecdotes only tenuously describe "nefarious" elements. You cannot encrypt a blood stain or a fingerprint, nor can you use an anonymity system to hide the fact that a known terrorist group is increasing the volume of data it sends and receives.
So what nefarious elements do you think are going to avoid prosecution by using crypto? A few years ago, there was a case of a group of child molesters using PGP and anonymous remailers (and possibly other privacy technologies) to communicate and exchange photos of themselves abusing children over Usenet. They were caught, arrested, and prosecuted following a well-coordinated investigation -- otherwise known as "good police work." Only a handful managed to avoid identification or prosecution, and it is worth noting that this was the case with investigations of criminal organizations long before good encryption was widely available. It should hardly be surprising that such a group was ultimately caught: they were sending each other photographic evidence of their own crimes.
It is also worth pointing out that anonymity systems are used by the police to catch criminals. In the case described above, the police used the very anonymity system that the child abusers were using, and were thus able to observe their messages while maintaining the secrecy of their investigation. Similar scenarios have played out with Tor. Criminals who use the Internet will eventually figure out which IP addresses belong to the police; it will be critical for the police to use anonymity systems to mask their IP addresses (this, in fact, is closely related to the reason the Navy created Tor in the first place). It is not so much that society's nefarious elements are more empowered now; it is more that the nature of the game has changed, that new tactics and strategies will be developed by both criminals and the police to utilize and cope with these systems.
I would argue that crypto and anonymity systems have benefited society more than they have harmed it. The same crypto and anonymity technologies used by the group I mentioned above have been used by political dissidents and activists to protect themselves from abusive governments; human rights activists have used such systems to protect the people they work with. Whistleblowers have used such systems in the past, and will have to do so in the future. For people who do not have an army at their command, these sorts of systems are necessary for their protection. If you were going to report Mafia activity to the police, would you rather use Tor/remailers/etc. or would you walk into the police station in person?
There is no such thing as a world of transparency and openness, because crime is part of human nature and because political ambition is part of human nature, and both of these behaviors exploit openness and transparency. I am all for an open government, but even I acknowledge that the government will need to keep some things secret -- military plans, investigations of dangerous criminal enterprises, the locations of witnesses to crimes, and many other things must be kept secret for society's benefit. Open and transparent government does not mean "secret free," it means "secrecy when it is absolutely necessary;" it is the responsibility of citizens to ensure that their representatives in the government are not declaring too many things to be secret, and it is the job of journalists to report to the citizens what is being kept secret from them. The sooner society realizes that, the better.