Live data from Hacker News

A Google bot scrapes pricing info by adding items to carts

wsj.com

281–290 of 291 posts

Re: A Google bot scrapes pricing info by adding items to carts

#281

Earlier quoted context omitted.

Perhaps news papers should stop running opinions and editorials altogether, or move them to an entirely different brand.

I don't really think that's the answer. Actual physical newspapers ran editorials / opinions for decades without a problem. I think the solution is better UI/UX.

A separate brand would be a better UX.

It's clear that internet distribution is very different from physical newspapers and people don't pay much attention to the domain and styling.

Re: A Google bot scrapes pricing info by adding items to carts

#282

Earlier quoted context omitted.

Well we can change this by including shipping as the total price and not give deals on shipping. Deals on shipping are dark patterns.

I don't understand. So if the item is $5 and shipping is $5 regardless of the number of items being bought, then if I bought 2 I should pay $20 instead of $15?

What don't you understand about deals on shipping being dark patterns?

Re: A Google bot scrapes pricing info by adding items to carts

#283

Earlier quoted context omitted.

I don't understand. So if the item is $5 and shipping is $5 regardless of the number of items being bought, then if I bought 2 I should pay $20 instead of $15?

What don't you understand about deals on shipping being dark patterns?

It seems that you are saying that properly charging for shipping to be a dark patterns? Or you mean different kind of deal? Can you elaborate? As far as I know, the shipping charge doesn't scale linearly with number of items, so to me including shipping within the items price is going to overcharge the customer.

Re: A Google bot scrapes pricing info by adding items to carts

#284

Protip: You will often get a discount coupon if you go through most of the checkout process(need to provide email), but wait a couple days. Many stores automate abandoned checkout promotions.

A good example of "proof of work" used for price differentiation.

Re: A Google bot scrapes pricing info by adding items to carts

#285

Earlier quoted context omitted.

Speaking with my physics hat on I have to caution against this -- I wouldn't call it entropy, and I am very happy abusing that word in a variety of other contexts. (Like at church I will call Satan entropy.) But the reason I'd balk here is that entropy as a phenomenon happens in absence of a driving force. (1) In the present WSJ case, this is not mere indifference on the part of readers: storytelling is deeply a part…

Damnn So why do air molecules not hit the ground ?

So they do—they just don’t remain there, they get kicked back up.

What is at stake is that you carry a subtle contradiction in your head, assuming you got some pieces of wisdom from your physics classes but not others.

One side of this is the minimum energy principle. Like, this is common sense, you leave a basketball bouncing in your driveway and you expect it to stop somewhere, probably (if it's not a perfectly flat blacktop) downhill of wherever you started it. Heck if you've had a hoop in your driveway you probably have a reflex to run after the ball when it touches ground, otherwise it'll eventually find the road and roll very far away as it chases the downhill. That's the minimum energy principle, dynamic friction reduces kinetic energy in a system while forces tend to make potential energy into kinetic energy, so you would expect if you just leave the system alone it ends up at rest at some minimum of potential energy.

The other side of the contradiction is that we tell you that energy is conserved cannot be created or destroyed. If you are very lucky, we tell you that there is a way of phrasing the laws of physics such that energy conservation is the same as saying that the laws of physics are the same today as they are tomorrow—we call this “time translation symmetry” and did the theorem that connects continuous symmetry is to conserved quantities is Noether’s Theorem if you are looking for something to google here.

The only way to resolve the contradiction is to say that friction is actually dissipation—energy is getting more spread out among the universe but is not being destroyed. So you want to picture a big bucket of water and a thousand little glasses and we empty the bucket only by putting it into all of those glasses. And the idea is that eventually if random processes take over the moving of the water from any of these to any other of these, all of them will have the same water level. You can actually see this if you see demonstrations of the siphon effect, water will actually flow up and down a hose to equalize two water levels in two reservoirs. When energy does this, has the same average occupation in every degree of freedom of a system, we say that the system has thermalized and we can measure its absolute temperature as that energy level. Technically temperature is not uniquely defined in any other context—mostly, we find physical objects whose properties like volume or length or so vary approximately linearly with temperature in this sense, then we use them as thermometers to measure temperatures in other contexts.

Now there is an interesting result, which is that if your bucket is at the same level as the cups, in some sense your bucket never ends up empty. Like there can be a lot of cups and that water can be spread over everything and there is only a tiny film of water in the actual bucket left, but it’s not zero.

This is also a theorem, it is called the fluctuation-dissipation theorem. It says that I can't dissipate energy into some environment without feeling noise from that environment prevent me from dissipating all of my energy into it: I have to accept random fluctuations back from it. In other words, there are no one-way channels for energy.

To bring this back to your question, the basketball only comes to rest on the ground because it has so much more energy than the thermal fluctuation energy which it gets back from the ground. When it eventually settles, it turns out that it is not fully at rest but is moving imperceptibly due to these fluctuations. And those fluctuations are imperceptible because the mass of the basketball is very large, large enough that this disturbs the center of mass by a height way smaller than the size of atoms, which it turns out are way smaller than the light you can see. So like even with a microscope, visible light is too chunky to show you this on a basketball.

But repeat the calculation for how far those 25 meV of thermal energy will launch a 28-amu nitrogen molecule and you will find that the height is roughly nine kilometers [1] which is a pretty good rough estimate for the height of the Earth's atmosphere, that's about where the troposphere ends. Just to be clear, the ground doesn't kick any individual molecules that high, they collide with other air molecules way before they get anywhere near that high, but that energy and momentum does ultimately get communicated to the whole swarm of air molecules and stops the swarm collectively from falling lower than that distance on average, even though everything is one big colliding mess. The first order prediction is actually an exponential decrease in density as you go to those higher heights, and I think that 9km figure is a 1/e decay constant, but the truth gets a lot more complicated as the ultraviolet light coming into the Earth is getting preferentially scattered in the high atmosphere and contributing a second source of energy to the system.

But yeah, the air doesn't fall down to the ground because the sun is shining and keeping our planet warm, and that warmth is imperceptible in the motion of a basketball but several kilometers in terms of the height of air molecules.

[1] https://www.google.com/search?q=25+meV+%2F+%2828+amu+*+9.81+...

Re: A Google bot scrapes pricing info by adding items to carts

#286

Earlier quoted context omitted.

> Genghis Khan Genghis Khan was neither a tyrant nor a monarch, nor was cutting off heads a part of his everyday life. He actually got more Mongol people more rights and a higher quality of life. I would request you to read a detailed and accurate account of his life by a historian. For example, Genghis Khan: His Conquests, His Empire, His Legacy by Frank McLynn https://amzn.com/B00X2ZW5ZI

Do you find the following inaccurate or biased, and if so, for the uninformed, briefly why? "Campaigns initiated in his lifetime include those against the Qara Khitai, Khwarezmia, and the Western Xia and Jin dynasties, and raids into Medieval Georgia, the Kievan Rus', and Volga Bulgaria. These campaigns were often accompanied by large-scale massacres of the civilian populations, especially in the Khwarazmian- and Wes…

> considered by many to have been a brutal ruler

He raided these places in military campaigns; he did not rule them at the time of the raids. So he was not a brutal ruler. In fact, he was a benevolent ruler, distributing loot equally and promoting on merit rather than nepotism.

As another commentator has pointed out, the brutality was an intimidation tactic. It was very selectively applied. https://news.ycombinator.com/item?id=23708182

He also did much to bring modern human rights to his subjects. You can read about it in Genghis Khan and the Making of the Modern World by Jack Weatherford, an anthropologist who lives in Mongolia part-time. https://amzn.com/B000FCK206

Re: A Google bot scrapes pricing info by adding items to carts

#287
post #155

Earlier quoted context omitted.

The site terms and conditions are displayed very publicaly and accessible to anyone who cares to read them. By entering your email address you are opting-in for transaction related emails, including new order confirmations, shipment notifications, and yes cart reminders. It's spelled out for you. It's the same for almost every ecommerce site. That's different than marketing emails, which require a separate explicit o…

> By entering your email address you are opting-in for transaction related emails You realize that you can't send marketing emails without double opt-in, right? "Hey, you forgot to buy these ..." is definitely a marketing email.

> You realize that you can't send marketing emails without double opt-in, right?

That's not true.

> "Hey, you forgot to buy these ..." is definitely a marketing email

Also untrue.

Re: A Google bot scrapes pricing info by adding items to carts

#288

This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process. I always have found that kind of shady but it's probably known to increase conversions. What I found interesting is that this an open attack vector for e-commerces. Multiple bots can hit a website and start addin…

As a developer of automotive ecommerce sites, it is very common for sites to list a higher price 0h the catalog, and show the true (generaly lower) price in the cart. This is becuase the businessmodels are highly margin sensative, and competative pricing can have big impact, so its a measure to try and mask real pricing.

Re: A Google bot scrapes pricing info by adding items to carts

#289

Earlier quoted context omitted.

Exactly this, the customer service for the average consumer from Amazon is very difficult to beat and is Google's biggest weakness. Bought some cables from Amazon Basic, one ended up not working, another had some cosmetic damage but works fine. They refunded both, sent out replacements, and just told me to discard them, it wasn't worth it for Amazon to pay to have it shipped back. Of course if you abuse this too much…

> Exactly this, the customer service for the average consumer from Amazon is very difficult to beat and is Google's biggest weakness. Amazon's customer service is a robot, which switches to someone in a callcenter in India, and then finally switches to a local person. I know because I recently had to contact them. Not sure how this is "difficult to beat".

It's difficult to beat because prices are a race to the bottom, and small players have no effective way to build up and manage reputations.

If I need a widget, and Vendor A charges a buck, while Vendor B charges two bucks, all else being equal, I'll buy from Vendor A. Bad customer service helps both vendors compete with each other, but prevents small companies, collectively, from competing with Amazon.

On eBay, small players do manage reputations, but only for a few weeks. If a product fails (or is discovered to be a fake) after 60 days, the seller is all good. Next sucker! There are things I'll buy there, but far more I won't.

Google itself has the problem that culturally, it relies on algorithms which know better than you do, and is not a service company. It does great tech, but holds human being outside of Google in open contempt. That's find for running a search engine, adwords, or gmail, but it crashes-and-burns for ecommerce.

Re: A Google bot scrapes pricing info by adding items to carts

#290
post #75

Earlier quoted context omitted.

How does that change by having the bot add items to the cart? You haven't solved anything You are still left with the same scenario as if the store listed the individual shipping price on the front page Google isn't going to know what other items you _might_ add to show you a "real" shipping cost

And what's even more interesting - human would do exactly the same thing. Add items to card, check the total price and then decide whether to buy it(i remember trying to order some stuff form one japanese plamo store - and it didn't provide exact prices before checkout. I went through the process, but even the cheapest option for delivery was way too high - as 2x price of the whole order)

I've had that sort of experience with a Japanese store once--the prices were good but the shipping (the only thing they offered was international FedEx) killed it. The US is just as bad--we don't have slow international options.
Post reply on HN