I had to build a web scraper to buy groceries • Hacker News Board

Re: I had to build a web scraper to buy groceries

#21

Post by meritt » Sun, Mar 29, 2020, 2:22 AM UTC

Nextgrid wrote: post #19

xzel wrote: post #7

Earlier quoted context omitted.

Web queues are ineffective as shown by sneaker bots. You’d need actual ID style authentication of accounts. If you’re talking about order queues than people will stuff the queue and cancel their orders or just have more hoarding if their multiple deliveries get processed. If it’s FIFO same technical advantages will apply: bots can click much faster than humans ever will. I think the issue is more human nature and thi…

Sneaker stores rely on exclusivity so the bots used by resellers are considered a problem. Of course, the real problem is the idea of exclusivity in the first place but that’s besides the point. Supermarkets don’t have that problem - their goods are not that exclusive and not sought after at scale (anyone wanting to resell the products at scale can get them cheaper from the manufacturer directly).

If it wasn't for the reselling market, there wouldn't be the same level of demand for sneakers. They're just shoes.

Re: I had to build a web scraper to buy groceries

#22

Post by wbl » Sun, Mar 29, 2020, 2:22 AM UTC

throwaway3563 wrote: post #20

jbm wrote: post #14

Earlier quoted context omitted.

38% of the people hospitalized with COVID-19 are between 20 and 54. [1] Everyone can catch the illness, everyone can wind up in the hospital, and depending on the state of the local hospitals, everyone is in immediate danger. [1] https://www.msn.com/en-ca/health/medical/close-to-40percent-...

All the more reason to dedicate the slots for the disproportionately-represented older cohort.

Or reorganize to have a central delivery service that goes to all the houses in an area.

Re: I had to build a web scraper to buy groceries

#23

Post by 6510 » Sun, Mar 29, 2020, 2:23 AM UTC

klipt wrote: post #18

Swizec wrote: post #6

I’ve just been going to the grocery store as usual and it works great. You show up in the morning and there’s no line yet and shelfs are full. Later in the day they meter entry so you can keep distance inside. Don’t be that ass who makes others do your shopping just coz you’re too afraid to.

In store: you need to have someone stock the shelves, someone walk through the store and pick groceries, and someone handle the payment in person. Chance of infection: depends how long since someone coughed in the store. Online: you need someone to stock the orders and wait for people to drive up for curbside pickup. Chance of infection: low. Surely the second option is both safer and more efficient?

If you work in a supermarket you know exactly where each product is. Standing in line is also a huge waste of time.

Re: I had to build a web scraper to buy groceries

#24

Post by sneak » Sun, Mar 29, 2020, 2:23 AM UTC

Brakenshire wrote: post #4

Young healthy people shouldn’t be doing this, the slots should go to the ill and elderly, people who are in immediate danger from catching the illness.

How not-young must one be to be legitimate in taking steps like this to avoid contracting this potentially fatal illness?

I’m not yet 40. I have bought all of my food online and had it delivered for years now. Should I now begin exposing myself to the infection risks posed by going inside a grocery store?

This may sound like snark but it’s not. I’m curious about the opinions of people who feel this way.

Re: I had to build a web scraper to buy groceries

#25

Post by 6510 » Sun, Mar 29, 2020, 2:23 AM UTC

Greasemonkey seems a much better weapon.

Re: I had to build a web scraper to buy groceries

#26

Post by xzel » Sun, Mar 29, 2020, 2:24 AM UTC

somebodynew wrote: post #16

xzel wrote: post #7

Earlier quoted context omitted.

Web queues are ineffective as shown by sneaker bots. You’d need actual ID style authentication of accounts. If you’re talking about order queues than people will stuff the queue and cancel their orders or just have more hoarding if their multiple deliveries get processed. If it’s FIFO same technical advantages will apply: bots can click much faster than humans ever will. I think the issue is more human nature and thi…

Bots are typically used to buy out a limited supply of something like concert tickets. Clicking quickly or responding immediately to availability changes wouldn't give bots any advantage when there's nothing but a FIFO queue. If I were setting this up for a grocery delivery platform, I would require the delivery address be provided upfront to enter the queue. Validate and standardize the address using the official Po…

But unfortunately there are a limited number of per day delivery slots, so bots will own anything FIFO. They will monitor the page and be the first ones in. There also aren’t really fully standardized addresses per se, it’s actually kinda weird, but what’s stopping anyone using the apartment number next to them and receiving the delivery when notified?

Re: I had to build a web scraper to buy groceries

#27

Post by zxcvbn4038 » Sun, Mar 29, 2020, 2:24 AM UTC

Brakenshire wrote: post #4

Young healthy people shouldn’t be doing this, the slots should go to the ill and elderly, people who are in immediate danger from catching the illness.

At this point everyone is in immediate danger of catching the virus, the ill and the elderly are just the ones less likely to recover from it.

Probably the fairest way to do it is everyone gets in the queue instead of competing for spots, and limits are enforced to help spread demand.

Re: I had to build a web scraper to buy groceries

#28

Post by gizmo686 » Sun, Mar 29, 2020, 2:25 AM UTC

jbm wrote: post #14

Brakenshire wrote: post #4

Young healthy people shouldn’t be doing this, the slots should go to the ill and elderly, people who are in immediate danger from catching the illness.

38% of the people hospitalized with COVID-19 are between 20 and 54. [1] Everyone can catch the illness, everyone can wind up in the hospital, and depending on the state of the local hospitals, everyone is in immediate danger. [1] https://www.msn.com/en-ca/health/medical/close-to-40percent-...

That 38% figure is not relevent in this context, because it does not control for the demographics of the population in general. 45% of admisions are over 65, despite the fact that >65 makes up only 16% of the total population [0].

Further, according the the underlying report, of the 22-44 year olds admitted to the hospital, only 2-4% went to the ICU. The 75–84 bracket had 11%–31% admitance to the ICU.

Of the deaths reported, 80% were over 65, while the other 20% was 20–64.

Yes, this virus can kill anyone, but statiscally speeking, it is far more likely to kill you when you are old. If there is a limited supply of a protective messure, we should prioritize giving it to the far more vulnerable.

[0] https://en.wikipedia.org/wiki/Demographics_of_the_United_Sta...

Re: I had to build a web scraper to buy groceries

#29

Post by Swizec » Sun, Mar 29, 2020, 2:29 AM UTC

klipt wrote: post #18

Swizec wrote: post #6

I’ve just been going to the grocery store as usual and it works great. You show up in the morning and there’s no line yet and shelfs are full. Later in the day they meter entry so you can keep distance inside. Don’t be that ass who makes others do your shopping just coz you’re too afraid to.

In store: you need to have someone stock the shelves, someone walk through the store and pick groceries, and someone handle the payment in person. Chance of infection: depends how long since someone coughed in the store. Online: you need someone to stock the orders and wait for people to drive up for curbside pickup. Chance of infection: low. Surely the second option is both safer and more efficient?

A lot of online options send a person who goes into the store and walks around to get all your stuff. Instacart for example.

Re: I had to build a web scraper to buy groceries

#30

Post by gruez » Sun, Mar 29, 2020, 2:29 AM UTC

xzel wrote: post #7

somebodynew wrote: post #3

It seems like a lot of demand issues highlighted by the pandemic could be solved by queues. Instead of saying "there are no slots available", put the user in the queue. Process the queue in order. The improvement in wait time and demand accuracy would be a win for both consumers and the business.

Web queues are ineffective as shown by sneaker bots. You’d need actual ID style authentication of accounts. If you’re talking about order queues than people will stuff the queue and cancel their orders or just have more hoarding if their multiple deliveries get processed. If it’s FIFO same technical advantages will apply: bots can click much faster than humans ever will. I think the issue is more human nature and thi…

>Web queues are ineffective as shown by sneaker bots. You’d need actual ID style authentication of accounts.

...and that's rendered ineffective by SaaS bot companies that take your account logins and payment details and places the orders on your behalf, for a fee of course[1]. The economically efficient solution would be to auction off the slots, so at least the grocer is reaping the profit rather than some third party. It also has the benefit of capitalizing the grocer so they can hire more people and/or buy more equipment so there are more slots in total. Also, if it bothers you that some FAANG programmer can afford deliveries than some old granny, this can be alleviated by reserving a certain % of the slots to "needy" people (eg. old, disabled, sick, immunocompromised) subject to validation.

[1] kinda like https://en.wikipedia.org/wiki/Domain_drop_catching