Removing my site from Google search
1–10 of 119 posts
Re: Removing my site from Google search
#2Why not use robots.txt instead of littering your html with googlebot instructions?
Re: Removing my site from Google search
#3Why not use robots.txt instead of littering your html with googlebot instructions?
Yes, pretty sure this is the way to go.
You can even tell which bots are allowed to index and not.
Re: Removing my site from Google search
#4Why not use robots.txt instead of littering your html with googlebot instructions?
Hi, author here. Google stopped supporting robots.txt [edit: as a way to fully remove your site] a few years ago, so these meta tags are now the recommended way of keeping their crawler at bay: https://developers.google.com/search/blog/2019/07/a-note-on-...
Re: Removing my site from Google search
#5Why not use robots.txt instead of littering your html with googlebot instructions?
Hi, author here. Google stopped supporting robots.txt [edit: as a way to fully remove your site] a few years ago, so these meta tags are now the recommended way of keeping their crawler at bay: https://developers.google.com/search/blog/2019/07/a-note-on-...
Did you actually read your link? That's not at all what it says.
Re: Removing my site from Google search
#6Earlier quoted context omitted.
Hi, author here. Google stopped supporting robots.txt [edit: as a way to fully remove your site] a few years ago, so these meta tags are now the recommended way of keeping their crawler at bay: https://developers.google.com/search/blog/2019/07/a-note-on-...
Did you actually read your link? That's not at all what it says.
To be clear, stopped supporting robots.txt noindex a few years ago.
Combined with the fact that Google might list your site [based only on third-party links][1], robots.txt isn't an effective way to remove your site from Google's results.
Sorry, could have been clearer.
[1]: https://developers.google.com/search/docs/advanced/robots/in...
Re: Removing my site from Google search
#7Earlier quoted context omitted.
Hi, author here. Google stopped supporting robots.txt [edit: as a way to fully remove your site] a few years ago, so these meta tags are now the recommended way of keeping their crawler at bay: https://developers.google.com/search/blog/2019/07/a-note-on-...
Did you actually read your link? That's not at all what it says.
>noindex in robots meta tags: Supported both in the HTTP response headers and in HTML, the noindex directive is the most effective way to remove URLs from the index when crawling is allowed.
Seems clear enough to me
Re: Removing my site from Google search
#8Earlier quoted context omitted.
Hi, author here. Google stopped supporting robots.txt [edit: as a way to fully remove your site] a few years ago, so these meta tags are now the recommended way of keeping their crawler at bay: https://developers.google.com/search/blog/2019/07/a-note-on-...
Did you actually read your link? That's not at all what it says.
Quote from the linked article:
“ For those of you who relied on the noindex indexing directive in the robots.txt file, which controls crawling, there are a number of alternative options:”
The first option is the meta tag. It does mention an alternative directive for robots.txt, however.
Re: Removing my site from Google search
#9Why not use robots.txt instead of littering your html with googlebot instructions?
Or even better, iptables rules :P
Re: Removing my site from Google search
#10Earlier quoted context omitted.
Hi, author here. Google stopped supporting robots.txt [edit: as a way to fully remove your site] a few years ago, so these meta tags are now the recommended way of keeping their crawler at bay: https://developers.google.com/search/blog/2019/07/a-note-on-...
Did you actually read your link? That's not at all what it says.
This page has a little more detail: https://developers.google.com/search/docs/advanced/crawling/...
"If other pages point to your page with descriptive text, Google could still index the URL without visiting the page. If you want to block your page from search results, use another method such as password protection or noindex. "