Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Google bot scrapes pricing info by adding items to carts

wsj.com

Post reply on HN
Print view View original

261–270 of 291 posts

Re: A Google bot scrapes pricing info by adding items to carts

#261

Post by kabacha »

AnthonyMouse wrote: post #191
axlee wrote: post #172

Earlier quoted context omitted.

If you want the "pure" news as in 1921, subscribe directly to AP/AFP/Reuters/EFE. The world has changed, and most of the added value of newspapers now comes from contextualization that you seem to disregard to having no value, despite being the higher journalistic effort. It's easy to report that a chicken died crossing the road, it's a lot harder to explain what chain of events led to a chicken dying crossing the ro…

> The world has changed, and most of the added value of newspapers now comes from contextualization that you seem to disregard to having no value, despite being the higher journalistic effort. But that's not what most of them are doing anymore. It's not so much context as it is agenda. The stories are written to lead you to a particular conclusion. From right now: NY Times: Republicans are abruptly pushing Americans…

Fox News isn't even a news source — of course they "inject agenda". As for NY Times I don't feel see any agendas in the titles, am I missing something?

Re: A Google bot scrapes pricing info by adding items to carts

#262

Post by kabacha »

yongjik wrote: post #97
YetAnotherNick wrote: post #90

Earlier quoted context omitted.

How do I use robots.txt to tell google to not add item to the shopping cart?

Erm... hide the shopping cart page behind robots.txt?

As someone who has seen way too many robots.txt files that's exactly how you do it.

Re: A Google bot scrapes pricing info by adding items to carts

#263

Post by pjc50 »

e15ctr0n wrote: post #160
gretch wrote: post #156

Earlier quoted context omitted.

This theory doesn't hold up against the test of time. We used to have tyrant monarchs - Genghis Khan would roam the steppe and cutting people's heads off and that would be normal every day life. There are still bad things today, but more people have more rights and a higher quality of life.

> Genghis Khan Genghis Khan was neither a tyrant nor a monarch, nor was cutting off heads a part of his everyday life. He actually got more Mongol people more rights and a higher quality of life. I would request you to read a detailed and accurate account of his life by a historian. For example, Genghis Khan: His Conquests, His Empire, His Legacy by Frank McLynn https://amzn.com/B00X2ZW5ZI

Wasn't expecting to see Khan revisionism here, but I suppose for every mass murderer there are people who like defending them.

Re: A Google bot scrapes pricing info by adding items to carts

#265

Post by datenhorst »

gruez wrote: post #89
the_pwner224 wrote: post #62

Earlier quoted context omitted.

On many websites it still is. But recently many smaller independent stores use the Shopify platform where shipping is the penultimate step, before billing. You have to give address, email(!), phone number(!, mandatory), etc. before getting the price. I normally just use a fake email and number to get shipping price, and then do the actual checkout in incognito. Pretty sure if you do enter your real info and don't con…

I'm pretty sure most sites (eg. Amazon) does the same thing. Probably for the reason you mentioned: so they can have your contact information to send you spam later.

My girlfriend uses this as a pretty effective tactic to get discounts - you just have to wait a couple of days and they'll send you an email with a lower price to keep down abandonment rates

Re: A Google bot scrapes pricing info by adding items to carts

#266

Post by AnthonyMouse »

ebg13 wrote: post #182
Youden wrote: post #135

When and why did news cease being news and start being short stories and opinion? This entire article could have been cut down to the last few paragraphs and nothing of value would have been lost. Look at The New York Times in 1921 [0]. Generally the stories are factual and to the point. The entire front page seems to be pure news. There's very little storytelling here, at most there are a few timelines of events. Lo…

I'm sure that it predates Fox News Channel by a lot, after all, news business are business first and news second, but it would be misguided to ignore the fact that FNC has completely dominated viewership numbers in the US almost since its inception by doing exactly what you're complaining about. Opinion is a format that engages large numbers of people and therefore makes money extremely well.

> FNC has completely dominated viewership numbers in the US almost since its inception by doing exactly what you're complaining about.

To be fair, they mostly dominate viewership numbers by being the only right-leaning news network available, so they get half the viewing public to themselves while their competitors fight over the other half.

Sort of an indictment of the efficient market hypothesis that nobody has figured that out and gone into competition with them.

Re: A Google bot scrapes pricing info by adding items to carts

#267

Post by galkk »

hnburnsy wrote: post #246
asdf21 wrote: post #221

Earlier quoted context omitted.

>The WSJ is one of the few quite reputable news rooms out there. Hahhahahah I think that myth evaporated when they spend months trying to "expose" Pewdiepie.

All the news doesn't have to be serious. Hope you didn't miss the WSJ's reporting on Theranos which potentially saved lives by exposing Theranos' fraudulent lab tests.

They tried very hard to kill guy’s reputation by calling him nazi and falsifying facts. Those news were supposed to be taken seriously.

I’d even say that they were taken seriously, but in the sense that WSJ didn’t expected.

Re: A Google bot scrapes pricing info by adding items to carts

#268

Post by eythian »

yongjik wrote: post #81

robots.txt, man, if you don't want search engines to visit certain part of your page, use robots.txt! Once heard a tale of an angry site owner calling Google (back when Google itself was novel) - Google deleted his whole website! Turned out he had "DELETE" button in each page, which generated plain GET request. So Googlebot visited the site, followed links to every page, and then of course followed every link that ge…

I think you're thinking of this: https://thedailywtf.com/articles/The_Spider_of_Doom - which obviously has two issues: the auth, and the actions on GET.

Re: A Google bot scrapes pricing info by adding items to carts

#269

Post by AnthonyMouse »

kabacha wrote: post #261
AnthonyMouse wrote: post #191

Earlier quoted context omitted.

> The world has changed, and most of the added value of newspapers now comes from contextualization that you seem to disregard to having no value, despite being the higher journalistic effort. But that's not what most of them are doing anymore. It's not so much context as it is agenda. The stories are written to lead you to a particular conclusion. From right now: NY Times: Republicans are abruptly pushing Americans…

Fox News isn't even a news source — of course they "inject agenda". As for NY Times I don't feel see any agendas in the titles, am I missing something?

> Fox News isn't even a news source — of course they "inject agenda".

You could say that about a lot of places these days.

> As for NY Times I don't feel see any agendas in the titles, am I missing something?

In the first case they're both covering the exact same story but putting the opposite spin on it. Trump recommends that people wear a mask but then he often doesn't wear one himself.

He says he doesn't wear it when he's only around people who have already been tested, which could be true enough but I suspect a big reason is that it impairs communication. Human communication is lossy and full of redundancy to compensate. If you don't quite hear something but you can see their lips moving you still know what they said. People say as much with facial expressions as with words. But not if you have a mask over your face.

Trump's primary job right now is communicating, so "get Trump to wear a mask whenever he's talking to the public" becomes a priority for anybody who doesn't like him, because they have a plausible-sounding line about him setting an example, so it's a win-win for them -- either they paint him as reckless for not wearing a mask or they get him to wear it which impairs his communication. The media writes a hundred stories about it a week. Fox is only even covering that story to try to defend him, the others are using it as a political weapon and they know it.

A mask also reminds everybody of the pandemic and a pandemic is always going to be bad for the incumbent. Which is the issue with the other story -- not the contents but the priority. Trump has been wanting to reopen the country, so reopening problems in some places are bad for Trump, so that's their top story. Important things happening in Hong Kong right now. I think there's some BLM stuff going on too.

It would be one thing if this was an anomaly, but it's consistent.

The Times literally has a thing on their site right now called "Meet the Supporters Trump has Lost."

Re: A Google bot scrapes pricing info by adding items to carts

#270

Post by adwww »

input_sh wrote: post #171
shultays wrote: post #152

Earlier quoted context omitted.

I find not including tax better tbh. That way people are being reminded how much tax they are with every purchase.

Every EU bill should come with the price without VAT (value-added tax) and with the VAT applied, leading to the same result. The only difference is that the VAT is a certain % depending on your country (mostly 15-25%), making it easy for a merchant to calculate it with a single data point. Percentage is also always a fixed round number, so you can calculate it in your head when you stumble upon "+ VAT" somewhere with…

The UK was 17.5% VAT for years.
Post reply on HN