http://archive.is/YRkQe
Thanks! I was not aware you could use Web Archive for that. All the more reason to Love that site!
A Google bot scrapes pricing info by adding items to carts
51–60 of 291 posts
Re: A Google bot scrapes pricing info by adding items to carts
#52This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. I always have found that kind of shady but it's probably known to increase conversions. What I found interesting is that this an open attack vector for e-commerces. Multiple bots can hit a website and start addin…
How do you show a final price if you don't have all the information needed: tax locality, shipping preference, total cart value discounts etc.
If I'm remembering right, Best Buy used to have "deals" on items that they "couldn't show you" until item was in cart. They may still be doing this. Best Buy's justification for it was that its agreements with manufacturers prevented it from displaying items below certain prices on their site. I'd never seen this elsewhere to know how pervasive these agreements were (or if Best Buy was just taking losses on certain items).
Re: A Google bot scrapes pricing info by adding items to carts
#53This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. I always have found that kind of shady but it's probably known to increase conversions. What I found interesting is that this an open attack vector for e-commerces. Multiple bots can hit a website and start addin…
Not all shipping charges can be calculated ahead of time. For example, you may offer free shipping on orders over $50. You may charge $9.99 for the first item, $5.99 for each additional item. You may charge by weight of the whole order. You may have oversized items or packages that can be combined to reduce shipping charges. Some items may ship together as OTR Freight, while others can go via the local postal service…
What this poster said.
Yes, a lot of smaller e-commerce platforms could do this, but finalizing order value can be a very complex workflow for bigger merchants with more varied sku mixes.
I’ve worked in multi-billion dollar Ecom companies where the programs to refine the order checkout process gets scoped as a multi year effort accounting for a couple of decades of legacy cruft... even if you separate the “product/tax/shipping” calculations from the “customer/credit/rewards” dependencies. But it’s often not worth separating them because they’re very inter-dependent. Moreso when you involve drop shipping or made-to-order things.
Re: A Google bot scrapes pricing info by adding items to carts
#54Earlier quoted context omitted.
> This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. It's usually not possible because you don't know how much the shipping + taxes are until the customer enters the billing information.
Yeah that's true. But from a UX perspective there are ways to make this less opaque. Perhaps a call to action at the top of the listing with an entry box to enter the Zip Code so an approximate final price can be calculated. Any good UX designer can come up with a solution for this in a couple of hours or less. There's just no motivation to make it happen because this obfuscation of data is particularly optimized and…
Amazon just lists it as a "subtotal," which I think is probably the best way to do it. I don't know about the "UX designer in a couple of hours" line: automatic shipping is a nightmarish bag of worms and isn't really a UX problem. What do you do if they order multiple SKU's that don't pack nicely into one box or are warehoused in different locations? Or if there is something weird about their shipping address? What do you use for box size and weight, and what approximation did you use for dunnage?
You can approximate UPS/FedEX costs by fitting a trend line, they are decently modeled by a linear (base charge + K*distance) function, but when you go to buy the label you might be way off. This puts you in the lose-lose-lose of either eating the difference of wrong estimates, overcharging for shipping and losing conversions, or just making people hate you by increasing the shipping over the estimate. Making a shipping API call is noticeably slow, so most people require an interaction after the address is entered.
Tim Sweeney's hot take was that "the two hardest problems in computer science are cache invalidation and shopping carts!"
Re: A Google bot scrapes pricing info by adding items to carts
#55It's just price data collection. In particular, MAP policies can be skirted by not publishing a final price but having a price below MAP in the cart which is a common tactic that online sellers utilize. By pretending to walk through the cart, all sorts of data about pricing, taxes, etc. can be learned. It's not entirely uncommon to see different prices at different times, for different user agents, for different loca…
MAP == Minimum Advertised Price
Re: A Google bot scrapes pricing info by adding items to carts
#56Earlier quoted context omitted.
How do you show a final price if you don't have all the information needed: tax locality, shipping preference, total cart value discounts etc.
If I'm remembering right, Best Buy used to have "deals" on items that they "couldn't show you" until item was in cart. They may still be doing this. Best Buy's justification for it was that its agreements with manufacturers prevented it from displaying items below certain prices on their site. I'd never seen this elsewhere to know how pervasive these agreements were (or if Best Buy was just taking losses on certain i…
> Best Buy used to have "deals" on items that they "couldn't show you" until item was in cart. They may still be doing this. Best Buy's justification for it was that its agreements with manufacturers prevented it from displaying items below certain prices on their site. I'd never seen this elsewhere to know how pervasive these agreements were
This was also pretty common on Amazon.
Re: A Google bot scrapes pricing info by adding items to carts
#57Re: A Google bot scrapes pricing info by adding items to carts
#58Would it be too much for Google to program the bot to get the final price, and then delete all the items from the cart? Seems rather rude, even for Google.
I abandon carts more often than not. Pretty much for the same reason as the bot: I wanna know how much I'm actually getting charged with taxes, shipping and coupons. I'll do similar orders on multiple stores, and only finalize the best deal if I'm satisfied with it. Sometimes I just "walk away" because nobody's selling at my pricepoint.
Is this rude? I really don't care.
Re: A Google bot scrapes pricing info by adding items to carts
#59Are there legal implications to Google bots transacting with websites under false pretenses? I mean their normal web crawler identifies itself as such. Here, I feel like they're committing (very) minor fraud by putting in fake shopper information and actively hiding their identity. Not a big deal if it were just some Joe Schmoe somewhere, but at their scale might it border on harassment? The robot equivalent of a pra…
You should worry more about sellers engaging in anti-competitive behavior like bait-and-switch or price fixing.
Re: A Google bot scrapes pricing info by adding items to carts
#60This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. I always have found that kind of shady but it's probably known to increase conversions. What I found interesting is that this an open attack vector for e-commerces. Multiple bots can hit a website and start addin…
Not all shipping charges can be calculated ahead of time. For example, you may offer free shipping on orders over $50. You may charge $9.99 for the first item, $5.99 for each additional item. You may charge by weight of the whole order. You may have oversized items or packages that can be combined to reduce shipping charges. Some items may ship together as OTR Freight, while others can go via the local postal service…
that's true that the calculations can get complicated pretty quickly in ecommerce, but google probably has all the data it needs (origin zip, destination zip, likely carrier(s), possibly even the weight/size of each item) to provide a pretty good estimate in most cases. they could even calculate a range for [1 item per box, all items in 1 box].
the important bit is to present it as a separate line item (with grand total) so that consumers can decide how much to trust the estimate.
that would be an even clearer shot across the bow of amazon, walmart, and the like, who provide comparison across their own platform merchants, but not across all merchants everywhere.