A Google bot scrapes pricing info by adding items to carts • Hacker News Board

Re: A Google bot scrapes pricing info by adding items to carts

#221

Post by asdf21 » Thu, Jul 02, 2020, 2:39 AM UTC

supernova87a wrote: post #174

Youden wrote: post #135

When and why did news cease being news and start being short stories and opinion? This entire article could have been cut down to the last few paragraphs and nothing of value would have been lost. Look at The New York Times in 1921 [0]. Generally the stories are factual and to the point. The entire front page seems to be pure news. There's very little storytelling here, at most there are a few timelines of events. Lo…

Maybe you don't know this, but the "A-hed" article of the WSJ is the humorous, light-hearted take on some cultural phenomenon that appears every couple of days. It's got a distinct separation (graphically) from the rest of the news, and is written not to be taken too seriously. (It's not so apparent in the online version, if you haven't read it before). So you don't have to worry that it's some broad decline in journ…

>The WSJ is one of the few quite reputable news rooms out there.

Hahhahahah

I think that myth evaporated when they spend months trying to "expose" Pewdiepie.

Re: A Google bot scrapes pricing info by adding items to carts

#222

Post by harry8 » Thu, Jul 02, 2020, 2:41 AM UTC

SquishyPanda23 wrote: post #219

amadeuspagel wrote: post #216

Earlier quoted context omitted.

Murdoch bought the WSJ in 2007. When is he going to start destroying its credibility "utterly"?

Uh have you read their commentary/opinions? Half the time they come off as if they're trolling. I'm sure at one point they were a thinking man's newspaper. At this point they're just fan service for people who have drunk the koolaid but can't stomach Fox's mass market approach.

To be fair the WSJ has always had some pretty outlandish opinion pieces. The tradition was that these were separate to the news reporting and the news reporting was untouched by them. But now it's in Murdoch stable. Sad.

Re: A Google bot scrapes pricing info by adding items to carts

#223

Post by URSpider94 » Thu, Jul 02, 2020, 2:44 AM UTC

Youden wrote: post #135

When and why did news cease being news and start being short stories and opinion? This entire article could have been cut down to the last few paragraphs and nothing of value would have been lost. Look at The New York Times in 1921 [0]. Generally the stories are factual and to the point. The entire front page seems to be pure news. There's very little storytelling here, at most there are a few timelines of events. Lo…

This is an example of what’s called literary journalism. Some people (including myself) consider this a high art form. If you want to study up on the classics, read Joan Didion and John McPhee - and Tom Wolfe, the titan of creative nonfiction.

There is no crime in making a creative story that people enjoy reading out of a true event.

Re: A Google bot scrapes pricing info by adding items to carts

#224

Post by dheera » Thu, Jul 02, 2020, 2:58 AM UTC

nickip wrote: post #180

dheera wrote: post #177

Earlier quoted context omitted.

You could GeoIP the user's IP address and display an initial tax + shipping estimate.

Tax isn't always only region based. You also need to account for Vat Tax which has a bunch of conditions around it too that you can't assume.

Include the statistical average tax in the price, and then up it or down it slightly at check out like a tax return. The user will be happy that the advertised price is a good estimate of the final price. Same for shipping.

Re: A Google bot scrapes pricing info by adding items to carts

#225

Post by bduerst » Thu, Jul 02, 2020, 2:59 AM UTC

rabanne wrote: post #111

inetknght wrote: post #23

Earlier quoted context omitted.

> Or quietly detect the bots and feed them junk data Just a few customers with alternative browsers being detected as "bots" will poison your reputation and income stream.

No, it definitely won’t. In fact our card payment system has been updated to think alternative os and browsers are high-risk and even decline payment. We didn’t have any revenue loss.

That's why bots now mimic users beyond user agents, even going so far as loading page assets and javascript. Unless you're using something like recaptcha V3, it's going to be difficult to detect them, and even that requires some interactions first.

Re: A Google bot scrapes pricing info by adding items to carts

#226

Post by randomdude402 » Thu, Jul 02, 2020, 3:07 AM UTC

mkl wrote: post #113

whoisjuan wrote: post #8

This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. I always have found that kind of shady but it's probably known to increase conversions. What I found interesting is that this an open attack vector for e-commerces. Multiple bots can hit a website and start addin…

It seems like it will also mess up item availability information. If an item has limited stock, bots adding it to carts could make it appear out of stock to real customers.

Most sites don't subtract it from inventory until it's on a completed order.

The main exceptions I can think of are venue and plane tickets, and hotel rooms. These might put a hold on a specific piece of inventory for a short time. They usually tell you when they do.

Re: A Google bot scrapes pricing info by adding items to carts

#227

Post by randomdude402 » Thu, Jul 02, 2020, 3:09 AM UTC

inetknght wrote: post #23

kqr wrote: post #17

Earlier quoted context omitted.

Or quietly detect the bots and feed them junk data after they've gone through the hoops. Not saying it's the better option, but knowing the business maybe the more likely one.

> Or quietly detect the bots and feed them junk data Just a few customers with alternative browsers being detected as "bots" will poison your reputation and income stream.

Nah. Google doesn't even let people log into their own email with alternative browsers, and they are doing fine.

Re: A Google bot scrapes pricing info by adding items to carts

#228

Post by lmm » Thu, Jul 02, 2020, 3:14 AM UTC

shultays wrote: post #152

miohtama wrote: post #125

Earlier quoted context omitted.

All EU prices must be tax included. All hospitality prices must include cleaning and service fees. It is only the US were hidden fees and charges may apply and the price regulation, if any, is more tilted towards corporations. Only upfront shipping charge is tricky because it depends on so many factors.

I find not including tax better tbh. That way people are being reminded how much tax they are with every purchase.

We don't break out how much the purchase price was reduced by the use of public goods, so it seems like breaking out the tax would be more misleading than informative.

Re: A Google bot scrapes pricing info by adding items to carts

#229

Post by lmm » Thu, Jul 02, 2020, 3:19 AM UTC

shadowgovt wrote: post #71

the_pwner224 wrote: post #65

Earlier quoted context omitted.

Probably a violation of the CFAA. Lots of people hate it because they think it's overreaching, and lots of companies use it to legally threaten scrapers and security research. But in this case Google is doing mass unauthorized use of other people's computers.

If I'm doing price comparison between online vendors, I will---as a human---put some items in the cart and get right to the edge of checkout to determine what my final bill would be. I may not close the sale if I'm looking at a better option elsewhere. How is what I'm doing materially different from what Google's doing? Is scale a factor that matters for CFAA?

Maybe you are violating the CFAA by doing that? It's a very broad law.

Re: A Google bot scrapes pricing info by adding items to carts

#230

Post by st1ck » Thu, Jul 02, 2020, 3:34 AM UTC

withinboredom wrote: post #77

maltelandwehr wrote: post #50

Earlier quoted context omitted.

„Messing up your competitors A/B test“ is not unheard of as a tactic in highly competitive ecommerce settings.

Do software engineers actually implement that? That seems pretty immoral. I'd rather let them run the a/b test and steal whatever solution they end up with.

I can't find reasons why would this be immoral. I'd say it's rather aggressive and won't earn you good reputation for sure. But it's sort of fair game. Compared to many business practices (lobbying, forced arbitration, patent trolling, DMCA, price dumping etc.) this is extremely mild one.