Controlling crawling and indexing now documented on code.google.com

googlewebmastercentral.blogspot.com