Let's imagine I have my online stock linked to limited physical items/assets, ex tickets for a show, which will get reserved for a period of time. This will be preventing genuine clients from buying them.
A Google bot scrapes pricing info by adding items to carts
91–100 of 291 posts
Re: A Google bot scrapes pricing info by adding items to carts
#92Earlier quoted context omitted.
It's not rude at a consumer level, where (in general) you're at least considering making the purchase. It's arguably rude at a bot level, depending on the frequency, where there is 0% chance of conversion.
The entire purpose of the bot is to provide listings to consumers who are looking to buy. If it was consumer journalist doing it to get the price for a news article (in a for-profit publication) about the product, would it be “rude”? If not, how is it for Google bot?
I didn't read the article due to the paywall, but I assume that the problem is that the problem is that these goods are reserved for that (non)-customer until the shopping cart times out? That is directly costing the merchant money, either in lost sales or having to maintain extra inventory.
So yeah, that bot really should have been programmed to end the session with an empty basket one way or another.
Re: A Google bot scrapes pricing info by adding items to carts
#93robots.txt, man, if you don't want search engines to visit certain part of your page, use robots.txt! Once heard a tale of an angry site owner calling Google (back when Google itself was novel) - Google deleted his whole website! Turned out he had "DELETE" button in each page, which generated plain GET request. So Googlebot visited the site, followed links to every page, and then of course followed every link that ge…
Re: A Google bot scrapes pricing info by adding items to carts
#94Genuine question, is this not considered a DoS attack? Let's imagine I have my online stock linked to limited physical items/assets, ex tickets for a show, which will get reserved for a period of time. This will be preventing genuine clients from buying them.
Re: A Google bot scrapes pricing info by adding items to carts
#95For people saying this to calculate the final price with shipping and tax, it's not (or at least not entirely). It is for this new sales conversion dark pattern where prices aren't listed until you add to cart. Ebay sellers are particularly bad offenders: https://www.ebay.com/itm/Open-Box-Certified-Samsung-Galaxy-1...
What am I missing on here? That item has the price listed without having to Add To Cart.
Re: A Google bot scrapes pricing info by adding items to carts
#96robots.txt, man, if you don't want search engines to visit certain part of your page, use robots.txt! Once heard a tale of an angry site owner calling Google (back when Google itself was novel) - Google deleted his whole website! Turned out he had "DELETE" button in each page, which generated plain GET request. So Googlebot visited the site, followed links to every page, and then of course followed every link that ge…
Re: A Google bot scrapes pricing info by adding items to carts
#97robots.txt, man, if you don't want search engines to visit certain part of your page, use robots.txt! Once heard a tale of an angry site owner calling Google (back when Google itself was novel) - Google deleted his whole website! Turned out he had "DELETE" button in each page, which generated plain GET request. So Googlebot visited the site, followed links to every page, and then of course followed every link that ge…
How do I use robots.txt to tell google to not add item to the shopping cart?
Re: A Google bot scrapes pricing info by adding items to carts
#98Genuine question, is this not considered a DoS attack? Let's imagine I have my online stock linked to limited physical items/assets, ex tickets for a show, which will get reserved for a period of time. This will be preventing genuine clients from buying them.
Possibly it is lower traffic than a full on dos?
Re: A Google bot scrapes pricing info by adding items to carts
#99robots.txt, man, if you don't want search engines to visit certain part of your page, use robots.txt! Once heard a tale of an angry site owner calling Google (back when Google itself was novel) - Google deleted his whole website! Turned out he had "DELETE" button in each page, which generated plain GET request. So Googlebot visited the site, followed links to every page, and then of course followed every link that ge…
How do I use robots.txt to tell google to not add item to the shopping cart?
But most online stores should be happy to have Google crawling their prices and showing up under the Shopping results.