Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Google bot scrapes pricing info by adding items to carts

wsj.com

Post reply on HN
Print view View original

21–30 of 291 posts

Re: A Google bot scrapes pricing info by adding items to carts

#21

Post by teh_klev »

ardy42 wrote: post #12

> When The Wall Street Journal contacted Google in June, a spokesman at the internet giant, after a few days of digging, provided an update: The mystery shopper is a bot of its own creation. > The purpose: making sure the all-in price for the product, including tax and shipping, matches the listing on its Google Shopping platform or in advertisements. It wasn’t to cause angst to merchants due to thousands of abandone…

> It reads like it was written by someone who hasn't fully mastered English

English is my first language, I didn't find the writing that terrible. It's certainly no worse than the standard of output from most large news outlets. Could you cite some examples?

Re: A Google bot scrapes pricing info by adding items to carts

#22

Post by neutronicus »

whoisjuan wrote: post #16
gruez wrote: post #14

Earlier quoted context omitted.

> This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. It's usually not possible because you don't know how much the shipping + taxes are until the customer enters the billing information.

Yeah that's true. But from a UX perspective there are ways to make this less opaque. Perhaps a call to action at the top of the listing with an entry box to enter the Zip Code so an approximate final price can be calculated. Any good UX designer can come up with a solution for this in a couple of hours or less. There's just no motivation to make it happen because this obfuscation of data is particularly optimized and…

e-commerce sites seem to be asking for my location all the time anyways.

If they're doing that anyways, they should have everything they need to hazard a pretty good guess (and then they have an actual inducement for me to provide it).

Re: A Google bot scrapes pricing info by adding items to carts

#23

Post by inetknght »

kqr wrote: post #17
bronzeage wrote: post #15

Earlier quoted context omitted.

Makes you wonder whether the smart thing to do is just make it convenient for the bots to get the info out so they won't ruin your data and waste your bandwidth. Can't fight them, join them. Can't really stop free information flow.

Or quietly detect the bots and feed them junk data after they've gone through the hoops. Not saying it's the better option, but knowing the business maybe the more likely one.

> Or quietly detect the bots and feed them junk data

Just a few customers with alternative browsers being detected as "bots" will poison your reputation and income stream.

Re: A Google bot scrapes pricing info by adding items to carts

#24

Post by ardy42 »

teh_klev wrote: post #21
ardy42 wrote: post #12

> When The Wall Street Journal contacted Google in June, a spokesman at the internet giant, after a few days of digging, provided an update: The mystery shopper is a bot of its own creation. > The purpose: making sure the all-in price for the product, including tax and shipping, matches the listing on its Google Shopping platform or in advertisements. It wasn’t to cause angst to merchants due to thousands of abandone…

> It reads like it was written by someone who hasn't fully mastered English English is my first language, I didn't find the writing that terrible. It's certainly no worse than the standard of output from most large news outlets. Could you cite some examples?

> English is my first language, I didn't find the writing that terrible. It's certainly no worse than the standard of output from most large news outlets. Could you cite some examples?

Sorry, it was my mistake. Before I saw the archive link, I had found a reposting on some sketchy website and got the tabs confused. That version is basically plagiarized but makes some weird word substitutions.

WSJ:

> For more than a year, online merchants selling items ranging from kayaks to keychains have puzzled over the mystery shopper with the generic name behind thousands of abandoned carts. Each cart has only one item.

https://apkmetro.com/who-is-the-mystery-shopper-leaving-behi...:

> For greater than a 12 months, on-line retailers promoting objects starting from kayaks to keychains have puzzled over the thriller shopper with the generic title behind hundreds of deserted carts. Every cart has just one merchandise.

WSJ:

> The purpose: making sure the all-in price for the product, including tax and shipping, matches the listing on its Google Shopping platform or in advertisements. It wasn’t to cause angst to merchants due to thousands of abandoned carts.

apkmetro plagiarization:

> The aim: ensuring the all-in worth for the product, together with tax and transport, matches the itemizing on its Google Purchasing platform or in commercials. It wasn’t to trigger angst to retailers as a consequence of hundreds of deserted carts.

Re: A Google bot scrapes pricing info by adding items to carts

#25

Post by inetknght »

bluGill wrote: post #20
ardy42 wrote: post #12

> When The Wall Street Journal contacted Google in June, a spokesman at the internet giant, after a few days of digging, provided an update: The mystery shopper is a bot of its own creation. > The purpose: making sure the all-in price for the product, including tax and shipping, matches the listing on its Google Shopping platform or in advertisements. It wasn’t to cause angst to merchants due to thousands of abandone…

They need to be non traceable. If I'm doing something underhanded with pricing information I want to detect Google and other such bots and give them different information.

You really think it's wise to lie to your customers?

Re: A Google bot scrapes pricing info by adding items to carts

#26

Post by whoisjuan »

neutronicus wrote: post #22
whoisjuan wrote: post #16

Earlier quoted context omitted.

Yeah that's true. But from a UX perspective there are ways to make this less opaque. Perhaps a call to action at the top of the listing with an entry box to enter the Zip Code so an approximate final price can be calculated. Any good UX designer can come up with a solution for this in a couple of hours or less. There's just no motivation to make it happen because this obfuscation of data is particularly optimized and…

e-commerce sites seem to be asking for my location all the time anyways. If they're doing that anyways, they should have everything they need to hazard a pretty good guess (and then they have an actual inducement for me to provide it).

Agree!

Re: A Google bot scrapes pricing info by adding items to carts

#28

Post by stronglikedan »

gruez wrote: post #14
whoisjuan wrote: post #8

This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. I always have found that kind of shady but it's probably known to increase conversions. What I found interesting is that this an open attack vector for e-commerces. Multiple bots can hit a website and start addin…

> This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. It's usually not possible because you don't know how much the shipping + taxes are until the customer enters the billing information.

I just used a website that had a simple form at the bottom of the cart. It had one text input for the postal code, and a button to get the rates to that postal code based on what was in my cart. IMHO, this is how it's done right, since all you need to know is general location and weight.

Re: A Google bot scrapes pricing info by adding items to carts

#29

Post by itronitron »

vmception wrote: post #2

That sparked a funny idea in my head, what if we tricked product managers industry wide to follow KPIs and A/B tests that resulted in a better user experience for consumers, instead of experiences that coincidentally slightly upticked "engagement". Because it seems like this mystery shopper is already doing that.

Given that engagement metrics have been heavily interfered with for many years, as a result of bots and other activities, and yet PMs still rely on them it seems unlikely that they will be pulled away from that spectacle anytime soon. I like your idea though.

Re: A Google bot scrapes pricing info by adding items to carts

#30

Post by bradlys »

justinwp wrote: post #27

Protip: You will often get a discount coupon if you go through most of the checkout process(need to provide email), but wait a couple days. Many stores automate abandoned checkout promotions.

Yes! This is also something that is common with smaller online retailers. Don't expect this with B&H, Adorama, or Newegg. Frequently these small companies give one time codes you won't see or be able to gain elsewhere.
Post reply on HN