Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Google bot scrapes pricing info by adding items to carts

wsj.com

Post reply on HN
Print view View original

61–70 of 291 posts

Re: A Google bot scrapes pricing info by adding items to carts

#61

Post by wdb »

gruez wrote: post #14
whoisjuan wrote: post #8

This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. I always have found that kind of shady but it's probably known to increase conversions. What I found interesting is that this an open attack vector for e-commerces. Multiple bots can hit a website and start addin…

> This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. It's usually not possible because you don't know how much the shipping + taxes are until the customer enters the billing information.

They could at least include the tax in the price. That's normally fixed depending on the item category e.g. low or high tax rate. Only the US is being weird with its taxes.

Re: A Google bot scrapes pricing info by adding items to carts

#62

Post by the_pwner224 »

jiofih wrote: post #40
stronglikedan wrote: post #28

Earlier quoted context omitted.

I just used a website that had a simple form at the bottom of the cart. It had one text input for the postal code, and a button to get the rates to that postal code based on what was in my cart. IMHO, this is how it's done right, since all you need to know is general location and weight.

Maybe I haven’t done enough online shopping recently, but as far back as I remember this used to be the norm: enter postcode to calculate shipping, get precise final price without even adding to cart. Is it not the case anymore?

On many websites it still is. But recently many smaller independent stores use the Shopify platform where shipping is the penultimate step, before billing. You have to give address, email(!), phone number(!, mandatory), etc. before getting the price. I normally just use a fake email and number to get shipping price, and then do the actual checkout in incognito. Pretty sure if you do enter your real info and don't continue with the purchase then you'll get email spam telling you to buy stuff.

Re: A Google bot scrapes pricing info by adding items to carts

#63

Post by jawns »

disposekinetics wrote: post #31
tacon wrote: post #18

Would it be too much for Google to program the bot to get the final price, and then delete all the items from the cart? Seems rather rude, even for Google.

Is abandoning a cart really rude behavior? I sometimes do it just to see if they'll spam me as a test of if I want to do business with a site.

It's not rude at a consumer level, where (in general) you're at least considering making the purchase. It's arguably rude at a bot level, depending on the frequency, where there is 0% chance of conversion.

Re: A Google bot scrapes pricing info by adding items to carts

#64

Post by hundchenkatze »

alpha_squared wrote: post #52
cglace wrote: post #47

Earlier quoted context omitted.

How do you show a final price if you don't have all the information needed: tax locality, shipping preference, total cart value discounts etc.

If I'm remembering right, Best Buy used to have "deals" on items that they "couldn't show you" until item was in cart. They may still be doing this. Best Buy's justification for it was that its agreements with manufacturers prevented it from displaying items below certain prices on their site. I'd never seen this elsewhere to know how pervasive these agreements were (or if Best Buy was just taking losses on certain i…

Newegg still does this. Here's an item from their 4th of July sale.

https://www.newegg.com/p/2AM-008Y-00003?Item=9SIAEG2BMZ4393

Re: A Google bot scrapes pricing info by adding items to carts

#65

Post by the_pwner224 »

rkagerer wrote: post #42

Are there legal implications to Google bots transacting with websites under false pretenses? I mean their normal web crawler identifies itself as such. Here, I feel like they're committing (very) minor fraud by putting in fake shopper information and actively hiding their identity. Not a big deal if it were just some Joe Schmoe somewhere, but at their scale might it border on harassment? The robot equivalent of a pra…

Probably a violation of the CFAA. Lots of people hate it because they think it's overreaching, and lots of companies use it to legally threaten scrapers and security research. But in this case Google is doing mass unauthorized use of other people's computers.

Re: A Google bot scrapes pricing info by adding items to carts

#67

Post by lalos »

whoisjuan wrote: post #8

This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. I always have found that kind of shady but it's probably known to increase conversions. What I found interesting is that this an open attack vector for e-commerces. Multiple bots can hit a website and start addin…

Also, seems like a clever hack for automated scraping after all most carts are pretty uniform in their structure.

Re: A Google bot scrapes pricing info by adding items to carts

#68

Post by bluntfang »

mmcconnell1618 wrote: post #43
whoisjuan wrote: post #8

This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. I always have found that kind of shady but it's probably known to increase conversions. What I found interesting is that this an open attack vector for e-commerces. Multiple bots can hit a website and start addin…

Not all shipping charges can be calculated ahead of time. For example, you may offer free shipping on orders over $50. You may charge $9.99 for the first item, $5.99 for each additional item. You may charge by weight of the whole order. You may have oversized items or packages that can be combined to reduce shipping charges. Some items may ship together as OTR Freight, while others can go via the local postal service…

Well we can change this by including shipping as the total price and not give deals on shipping. Deals on shipping are dark patterns.

Re: A Google bot scrapes pricing info by adding items to carts

#69

Post by ww520 »

wdb wrote: post #61
gruez wrote: post #14

Earlier quoted context omitted.

> This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. It's usually not possible because you don't know how much the shipping + taxes are until the customer enters the billing information.

They could at least include the tax in the price. That's normally fixed depending on the item category e.g. low or high tax rate. Only the US is being weird with its taxes.

Tax is depending on where the buyer is coming from.

Re: A Google bot scrapes pricing info by adding items to carts

#70

Post by hundchenkatze »

itronitron wrote: post #29
vmception wrote: post #2

That sparked a funny idea in my head, what if we tricked product managers industry wide to follow KPIs and A/B tests that resulted in a better user experience for consumers, instead of experiences that coincidentally slightly upticked "engagement". Because it seems like this mystery shopper is already doing that.

Given that engagement metrics have been heavily interfered with for many years, as a result of bots and other activities, and yet PMs still rely on them it seems unlikely that they will be pulled away from that spectacle anytime soon. I like your idea though.

> "and yet PMs still rely on them it seems unlikely that they will be pulled away from that spectacle anytime soon."

I think they meant, since PMs will never stop using metrics, we should write bots that skew those metrics in favor of an experience for the consumer rather than the perceived increase in engagement.

Post reply on HN