And just like that your wallets aren't anonymous anymore or at least are tied to each other beyond transactions between them. Seriously, though, this is kinda neat.
Wallets arent anonymous they are all in the block chain.
If someone uses a new Bitcoin address for each transaction, there is no way outside of statistical analysis (which may not even work in many cases) to know which addresses belongs to which wallets.