A Google bot scrapes pricing info by adding items to carts • Hacker News Board

Re: A Google bot scrapes pricing info by adding items to carts

#271

Post by [deleted] » Thu, Jul 02, 2020, 10:54 AM UTC

Alupis wrote: post #144

SquareWheel wrote: post #139

Earlier quoted context omitted.

>We send cart reminders to Abandoned Carts after a few days, sometimes with a coupon offer to complete checkout. I consider this spammy behaviour, and mark the emails as such. I can only hope this discourages such practices in the future.

It doesn't. If you mark it as Spam through most email programs, it's reported to the sender (Mandrill in our case) and Mandrill automatically black-lists your email address so we don't continue to send to someone that doesn't want the emails. That's a win-win.

[deleted]

Re: A Google bot scrapes pricing info by adding items to carts

#272

Post by amelius » Thu, Jul 02, 2020, 10:59 AM UTC

the_pwner224 wrote: post #65

rkagerer wrote: post #42

Are there legal implications to Google bots transacting with websites under false pretenses? I mean their normal web crawler identifies itself as such. Here, I feel like they're committing (very) minor fraud by putting in fake shopper information and actively hiding their identity. Not a big deal if it were just some Joe Schmoe somewhere, but at their scale might it border on harassment? The robot equivalent of a pra…

Probably a violation of the CFAA. Lots of people hate it because they think it's overreaching, and lots of companies use it to legally threaten scrapers and security research. But in this case Google is doing mass unauthorized use of other people's computers.

I think FTC should install a law that says that shops should be more transparent about their prices. That would solve the entire problem in the first place.

Re: A Google bot scrapes pricing info by adding items to carts

#273

Post by amelius » Thu, Jul 02, 2020, 11:01 AM UTC

ihumanable wrote: post #120

murgindrag wrote: post #115

Earlier quoted context omitted.

For Google (or anyone) to become a storefront for the entire web, they'd need to handle scams (and errors) well. eBay is a cesspool. Aliexpress is worse. Random web sites are bad. Amazon isn't perfect, but it's better. Amazon also has customer service; they've always made me whole. Random web sites, I'm basically SOL. Aliexpress and eBay are random. Someone flips a coin, heads seller wins, tails buyer wins, regardles…

Exactly this, the customer service for the average consumer from Amazon is very difficult to beat and is Google's biggest weakness. Bought some cables from Amazon Basic, one ended up not working, another had some cosmetic damage but works fine. They refunded both, sent out replacements, and just told me to discard them, it wasn't worth it for Amazon to pay to have it shipped back. Of course if you abuse this too much…

> Exactly this, the customer service for the average consumer from Amazon is very difficult to beat and is Google's biggest weakness.

Amazon's customer service is a robot, which switches to someone in a callcenter in India, and then finally switches to a local person. I know because I recently had to contact them.

Not sure how this is "difficult to beat".

Re: A Google bot scrapes pricing info by adding items to carts

#274

Post by jrockway » Thu, Jul 02, 2020, 11:14 AM UTC

harry8 wrote: post #214

supernova87a wrote: post #174

Earlier quoted context omitted.

Maybe you don't know this, but the "A-hed" article of the WSJ is the humorous, light-hearted take on some cultural phenomenon that appears every couple of days. It's got a distinct separation (graphically) from the rest of the news, and is written not to be taken too seriously. (It's not so apparent in the online version, if you haven't read it before). So you don't have to worry that it's some broad decline in journ…

> The WSJ is one of the few quite reputable news rooms out there. The WSJ is owned by Rupert Murdoch. The credibility of their newsroom begins being compromised by his owning it. He will destroy its credibility utterly by selling it for politicial influence in news reporting. Just as he has everywhere he has bought media. The particular example of compromised credibility that comes to mind is the Times of London whic…

I feel like he hasn't taken any actions to compromise the credibility of the Journal. Remember their Theranos coverage? Wikipedia writes: "Elizabeth Holmes asked Rupert Murdoch —- who at the time was a major investor in Theranos and owner of the Journal — to "personally kill" an investigative piece being written about Theranos. Murdoch refused, instead stating that he "had confidence in editors to handle the truth - whatever it may be". Murdoch went on to lose approximately $100 million in his investments in Theranos."

The opinion columns slant very far right, and I'm sure Murdoch has played some role in that. I subscribe to the Journal and am outraged every time I see them. But, it's good to see what other people are talking about. It does you no harm to read an opinion column that you vehemently disagree with. I'm sure people feel the same way about opinion columns in The New York Times, which I also read and tend to agree with.

My TL;DR here is that if you are interested in the news, you should get it from a variety of reputable sources. The Journal has proved over the years that it is reputable. I haven't seen anything recently that would change my mind. I have friends that get all their news through social media and they are constantly outraged based on pure misinformation. If they read the Journal, they'd have a much better understanding of the facts. (Same goes for the New York Times, or Washington Post, or The Guardian.)

Re: A Google bot scrapes pricing info by adding items to carts

#275

Post by dahfizz » Thu, Jul 02, 2020, 11:57 AM UTC

toomanybeersies wrote: post #239

dahfizz wrote: post #215

Earlier quoted context omitted.

> It's not so apparent in the online version, if you haven't read it before. I think this is the core issue that leads to sentiment like OPs. Real news still exists, but it's the highly editorialized and opinionated articles that are shared more widely. News agencies universally are terrible at obviously differentiating the two to users. 99% of people when they open an article scroll directly to the content. But any…

Is it the news agency's fault, or the reader's fault? In this case, it really isn't clear that it's not "real news". But I see plenty of people on social media sharing articles from news sites where they're clearly marked as an editorial or opinion piece, and believing or treating them as if they were exactly the same as a news article that attempts to paint a neutral picture of the facts. Do newspapers need to put a…

IMO, it is the news agency's responsibility to make it absolutely clear to the user what they are reading. If a majority of people come away with the wrong understanding after visiting a website, that is the website's fault. It's a clear UX problem.

Old paper newspapers accomplished this by physically having a separate section for opinion pieces. It's a harder problem when people share direct links to an article.

That said, news agencies don't try very hard. I think it's easy to argue they intentionally make it hard to differentiate. If WSJ really cared about making it clear to the user what this article was, they would have done more than hiding the word "A-hed" in a small font in the top corner.

Re: A Google bot scrapes pricing info by adding items to carts

#276

Post by dahfizz » Thu, Jul 02, 2020, 12:08 PM UTC

manigandham wrote: post #247

toomanybeersies wrote: post #239

Earlier quoted context omitted.

Is it the news agency's fault, or the reader's fault? In this case, it really isn't clear that it's not "real news". But I see plenty of people on social media sharing articles from news sites where they're clearly marked as an editorial or opinion piece, and believing or treating them as if they were exactly the same as a news article that attempts to paint a neutral picture of the facts. Do newspapers need to put a…

Perhaps news papers should stop running opinions and editorials altogether, or move them to an entirely different brand.

I don't really think that's the answer. Actual physical newspapers ran editorials / opinions for decades without a problem. I think the solution is better UI/UX.

Re: A Google bot scrapes pricing info by adding items to carts

#277

Post by cm2187 » Thu, Jul 02, 2020, 12:34 PM UTC

totony wrote: post #175

perl4ever wrote: post #165

Earlier quoted context omitted.

Do you find the following inaccurate or biased, and if so, for the uninformed, briefly why? "Campaigns initiated in his lifetime include those against the Qara Khitai, Khwarezmia, and the Western Xia and Jin dynasties, and raids into Medieval Georgia, the Kievan Rus', and Volga Bulgaria. These campaigns were often accompanied by large-scale massacres of the civilian populations, especially in the Khwarazmian- and Wes…

From what I've seen Ganghis Khan was brutal against non-surrendering nations, but beneficial for those who followed him. E.g. you woulsn't see him cutting his subjects heads, and his brutality was pretty much an intimidation tactic so others army would join him.

Oh it’s ok then

Re: A Google bot scrapes pricing info by adding items to carts

#278

Post by mitchdoogle » Thu, Jul 02, 2020, 2:10 PM UTC

tacon wrote: post #18

Would it be too much for Google to program the bot to get the final price, and then delete all the items from the cart? Seems rather rude, even for Google.

Do you consider visiting a site and then leaving the site before looking at anything there rude? People can't change their mind?

Putting something in a cart and leaving it there should be inconsequential to the seller. The only thing it might affect is their analytics. By the way, I consider analyzing my site visits and other data about me to be much more rude than abandoning a cart, especially when most sites don't even tell me they're doing it.

Re: A Google bot scrapes pricing info by adding items to carts

#279

Post by marcinzm » Thu, Jul 02, 2020, 2:46 PM UTC

amelius wrote: post #273

ihumanable wrote: post #120

Earlier quoted context omitted.

Exactly this, the customer service for the average consumer from Amazon is very difficult to beat and is Google's biggest weakness. Bought some cables from Amazon Basic, one ended up not working, another had some cosmetic damage but works fine. They refunded both, sent out replacements, and just told me to discard them, it wasn't worth it for Amazon to pay to have it shipped back. Of course if you abuse this too much…

> Exactly this, the customer service for the average consumer from Amazon is very difficult to beat and is Google's biggest weakness. Amazon's customer service is a robot, which switches to someone in a callcenter in India, and then finally switches to a local person. I know because I recently had to contact them. Not sure how this is "difficult to beat".

It's difficult to beat because those people can issue refunds. Now you need to deal with buyers scamming the system, sellers scamming the system, customer service employees scamming the system, and idiots. That's a lot of complexity to balance that can cost you a lot of money if not done well. In an industry with very low margins.

Re: A Google bot scrapes pricing info by adding items to carts

#280

Post by madmax108 » Thu, Jul 02, 2020, 3:06 PM UTC

I used to work at a company that provided APIs used for search/personalization/autosuggest for a whole bunch of huge e-commerce companies. Since the entire integration with the customer site was API based, we worked off of tracking pixels, API requests and cookies to determine shopping behaviour. A lot of this went into determining things like ranking (If someone searches "Tshirt" what shows up on the first page and in what order etc.)

Since we were only running search and not payment processing, the tracking pixel/API for "Add to Cart" was a big thing for us. The whole product ran on revenue-share so we were paid per X ATCs

Interesting to see if any of the customers were affected by bots doing ATC and how it was handled if it was.