Earlier quoted context omitted.
In the age of modern technology and the internet I think the sense and meaning of privacy is lost because everything is out there on the net waiting to be misused, abused, hacked, stolen, leaked or caught just like fish in the ocean. If I was US gov. I would've made law in the early 1990s that says every website, blog etc. is private property and no entity has right to crawl it if not given permission basically elimi…
> If I was US gov. I would've made law in the early 1990s that says every website, blog etc. is private property and no entity has right to crawl it if not given permission basically eliminating early Internet Search Engine industry meaning the behemoth like Google would probably never exist. it would have existed, just outside of the US
Robots.txt protocol exists but it is not even official internet standard under IETF. You can break robots.txt rules and there are no consequences at all. If robots.txt protocol was made law then it would be whole another story.