I am seeing files I upload to S3 in Google search results. I am not sure how Google found the URLs to these files as I've only shared them in email (Gmail).
Ask HN: Does Google Index “Secret” URLs Only Shared Through Gmail?
1–10 of 13 posts
Re: Ask HN: Does Google Index “Secret” URLs Only Shared Through Gmail?
#2You should put a robots.txt file in the S3 bucket if you don't want things in it indexed.
I (kind-of) doubt that they found the files through your e-mail, but the files in your S3 bucket are technically available for indexing since you don't have a robots.txt.
Re: Ask HN: Does Google Index “Secret” URLs Only Shared Through Gmail?
#3You should put a robots.txt file in the S3 bucket if you don't want things in it indexed. I (kind-of) doubt that they found the files through your e-mail, but the files in your S3 bucket are technically available for indexing since you don't have a robots.txt.
I do have a robots.txt, but for whatever reason its not found by Google
Re: Ask HN: Does Google Index “Secret” URLs Only Shared Through Gmail?
#4Go into Incognito and see if they are showing up there. Google might show you files from Google Drive and other places because you are logged into your Google Account, but it may not show up for anyone else.
Re: Ask HN: Does Google Index “Secret” URLs Only Shared Through Gmail?
#5Google definitely reads your email (gmail). As an example, if I have the word attachment written in my email and then attempt to send the email without any attachments, google will tell me that I used the word attachment in my email but didn't attach anything.
Re: Ask HN: Does Google Index “Secret” URLs Only Shared Through Gmail?
#6Google definitely reads your email (gmail). As an example, if I have the word attachment written in my email and then attempt to send the email without any attachments, google will tell me that I used the word attachment in my email but didn't attach anything.
"Reads your email" is ambiguous. All unencrypted email must be read just to transmit it over the network, or to store it on disk. Sounds silly, but the example you gave is actually even more benign: It could be done with Javascript that never sends anything across the network.
Re: Ask HN: Does Google Index “Secret” URLs Only Shared Through Gmail?
#7Am also facing the same problem.Anyone, know the answer?http://emailsupportnumber.org/yahoo-mail-customer-support-nu...
Re: Ask HN: Does Google Index “Secret” URLs Only Shared Through Gmail?
#8[deleted]
Re: Ask HN: Does Google Index “Secret” URLs Only Shared Through Gmail?
#9You should put a robots.txt file in the S3 bucket if you don't want things in it indexed. I (kind-of) doubt that they found the files through your e-mail, but the files in your S3 bucket are technically available for indexing since you don't have a robots.txt.
I do have a robots.txt, but for whatever reason its not found by Google
Are you using a custom domain? Robots.txt must be in your root folder, doesn't work with S3 urls.