Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I had to build a web scraper to buy groceries

utkuufuk.com

Post reply on HN
Print view View original

31–40 of 175 posts

Re: I had to build a web scraper to buy groceries

#31

Post by gruez »

xzel wrote: post #26
somebodynew wrote: post #16

Earlier quoted context omitted.

Bots are typically used to buy out a limited supply of something like concert tickets. Clicking quickly or responding immediately to availability changes wouldn't give bots any advantage when there's nothing but a FIFO queue. If I were setting this up for a grocery delivery platform, I would require the delivery address be provided upfront to enter the queue. Validate and standardize the address using the official Po…

But unfortunately there are a limited number of per day delivery slots, so bots will own anything FIFO. They will monitor the page and be the first ones in. There also aren’t really fully standardized addresses per se, it’s actually kinda weird, but what’s stopping anyone using the apartment number next to them and receiving the delivery when notified?

>But unfortunately there are a limited number of per day delivery slots

I guess the idea is to put everyone in one queue, rather than having separate queues per day/slot. Presumably this isn't an issue since everyone's on lockdown anyways, so they're always available to accept a delivery.

Re: I had to build a web scraper to buy groceries

#32

Post by pc86 »

jbm wrote: post #14
Brakenshire wrote: post #4

Young healthy people shouldn’t be doing this, the slots should go to the ill and elderly, people who are in immediate danger from catching the illness.

38% of the people hospitalized with COVID-19 are between 20 and 54. [1] Everyone can catch the illness, everyone can wind up in the hospital, and depending on the state of the local hospitals, everyone is in immediate danger. [1] https://www.msn.com/en-ca/health/medical/close-to-40percent-...

...which means 62% are people 55 or older. I'm not sure you're proving your point as well as you think you are.

Re: I had to build a web scraper to buy groceries

#33

Post by cortesoft »

xzel wrote: post #26
somebodynew wrote: post #16

Earlier quoted context omitted.

Bots are typically used to buy out a limited supply of something like concert tickets. Clicking quickly or responding immediately to availability changes wouldn't give bots any advantage when there's nothing but a FIFO queue. If I were setting this up for a grocery delivery platform, I would require the delivery address be provided upfront to enter the queue. Validate and standardize the address using the official Po…

But unfortunately there are a limited number of per day delivery slots, so bots will own anything FIFO. They will monitor the page and be the first ones in. There also aren’t really fully standardized addresses per se, it’s actually kinda weird, but what’s stopping anyone using the apartment number next to them and receiving the delivery when notified?

Just let people order far in advance then?

Re: I had to build a web scraper to buy groceries

#34

Post by banana_giraffe »

I had to do the same thing to get an Amazon Fresh order. Well, not quite the same thing, I ended up writing an Selenium project that automated the Amazon Fresh checkout process till it got to a delivery window page that didn’t complain there were no slots available.

If I can’t get groceries, it’s not the end of the world for me, since I can call on family if necessary, but I’d rather not make their stressful life any worse right now, since I’m not the only one in my family’s circle that needs help.

There are no great answers, since I might have taken a slot from someone that can’t write a web tool to scrape Amazon, and doesn’t have a family to fall back on. It sucks on many levels.

Re: I had to build a web scraper to buy groceries

#35

Post by ct520 »

I couldnt find an appointment slot with safeway this past week. Ended up writing a quick script on lunch that did the trick. Ran it as a schedule task to run every 5 mins. Pretty hacky and unpolished but worked, got a text message a day later and checked out. Not sure if anyone could use it but here it is. https://github.com/christru/safeway-delivery-check

Re: I had to build a web scraper to buy groceries

#36

Post by [deleted] »

gruez wrote: post #17
jbm wrote: post #14

Earlier quoted context omitted.

38% of the people hospitalized with COVID-19 are between 20 and 54. [1] Everyone can catch the illness, everyone can wind up in the hospital, and depending on the state of the local hospitals, everyone is in immediate danger. [1] https://www.msn.com/en-ca/health/medical/close-to-40percent-...

>38% of the people hospitalized with COVID-19 are between 20 and 54. [1] This is odd statistic to give, considering it's not normalized by how many 20-54 year olds are there. So I clicked through to the original study[1] and found a more helpful figure: people between the ages of 20-44 have a hospitalization rate of 14.3%-20.8%, and an ICU admission rate of 2.0%-4.2%. [1] https://www.cdc.gov/mmwr/volumes/69/wr/mm6912…

[deleted]

Re: I had to build a web scraper to buy groceries

#37

Post by gruez »

meritt wrote: post #21
Nextgrid wrote: post #19

Earlier quoted context omitted.

Sneaker stores rely on exclusivity so the bots used by resellers are considered a problem. Of course, the real problem is the idea of exclusivity in the first place but that’s besides the point. Supermarkets don’t have that problem - their goods are not that exclusive and not sought after at scale (anyone wanting to resell the products at scale can get them cheaper from the manufacturer directly).

If it wasn't for the reselling market, there wouldn't be the same level of demand for sneakers. They're just shoes.

>If it wasn't for the reselling market, there wouldn't be the same level of demand for sneakers

You can say that about literally anything. The value of anything goes up if it's liquid.

Re: I had to build a web scraper to buy groceries

#38

Post by toast0 »

xzel wrote: post #7
somebodynew wrote: post #3

It seems like a lot of demand issues highlighted by the pandemic could be solved by queues. Instead of saying "there are no slots available", put the user in the queue. Process the queue in order. The improvement in wait time and demand accuracy would be a win for both consumers and the business.

Web queues are ineffective as shown by sneaker bots. You’d need actual ID style authentication of accounts. If you’re talking about order queues than people will stuff the queue and cancel their orders or just have more hoarding if their multiple deliveries get processed. If it’s FIFO same technical advantages will apply: bots can click much faster than humans ever will. I think the issue is more human nature and thi…

Grocery delivery needs an address. You could pretty easily limit to one queued order per address, and limit order changes.

Re: I had to build a web scraper to buy groceries

#39

Post by gruez »

toomuchtodo wrote: post #9
xzel wrote: post #7

Earlier quoted context omitted.

Web queues are ineffective as shown by sneaker bots. You’d need actual ID style authentication of accounts. If you’re talking about order queues than people will stuff the queue and cancel their orders or just have more hoarding if their multiple deliveries get processed. If it’s FIFO same technical advantages will apply: bots can click much faster than humans ever will. I think the issue is more human nature and thi…

Require SMS or callback confirmation of the order slot using a non-Google/Twilio/SIP number (mobile phones or landlines both work here). Provide for one time order cancellation at no cost, cancellations after that incur a $10 cancellation fee. This should negate any technical advantage from a sophisticated entity. Queues still work, if managed properly.

>Require SMS or callback confirmation of the order slot using a non-Google/Twilio/SIP number (mobile phones or landlines both work here).

How does this deal with bots that places orders on other people's behalf?

https://news.ycombinator.com/item?id=22715875

Re: I had to build a web scraper to buy groceries

#40

Post by chris_va »

gruez wrote: post #30
xzel wrote: post #7

Earlier quoted context omitted.

Web queues are ineffective as shown by sneaker bots. You’d need actual ID style authentication of accounts. If you’re talking about order queues than people will stuff the queue and cancel their orders or just have more hoarding if their multiple deliveries get processed. If it’s FIFO same technical advantages will apply: bots can click much faster than humans ever will. I think the issue is more human nature and thi…

>Web queues are ineffective as shown by sneaker bots. You’d need actual ID style authentication of accounts. ...and that's rendered ineffective by SaaS bot companies that take your account logins and payment details and places the orders on your behalf, for a fee of course[1]. The economically efficient solution would be to auction off the slots, so at least the grocer is reaping the profit rather than some third par…

This assumes it's easy to resell a slot, which may not be correct (eg if you have to put in your delivery address when entering the queue). Tickets are a very different commodity.
Post reply on HN