Few months ago someone had posted open source scraper that got 1k+ votes (or maybe not) - it was used by NYTimes for realtime alerts.
Someone remind me please.
I had to build a web scraper to buy groceries
71–80 of 175 posts
Re: I had to build a web scraper to buy groceries
#72I had to do the same thing to get an Amazon Fresh order. Well, not quite the same thing, I ended up writing an Selenium project that automated the Amazon Fresh checkout process till it got to a delivery window page that didn’t complain there were no slots available. If I can’t get groceries, it’s not the end of the world for me, since I can call on family if necessary, but I’d rather not make their stressful life any…
Anytime I try to automate/scrape Amazon I get captcha'ed after a 1-2 attempts. Anything special you're doing with selenium?
Re: I had to build a web scraper to buy groceries
#73I had to do the same thing to get an Amazon Fresh order. Well, not quite the same thing, I ended up writing an Selenium project that automated the Amazon Fresh checkout process till it got to a delivery window page that didn’t complain there were no slots available. If I can’t get groceries, it’s not the end of the world for me, since I can call on family if necessary, but I’d rather not make their stressful life any…
I have made Amazon Fresh bot doing exactly the same using puppeteer.
Re: I had to build a web scraper to buy groceries
#74Of all the grocery companies in my area, only Walmart has it right: Choose an available pick up time, then start online shopping. All the others let you fill your cart, then at checkout tell you there are no slots available until the end of time. Waste of time, energy, and hope. F U, Target, Safeway, Albertsons, and Kroger.
Walmart is also tied for my top spot for in-person shopping during this. Walmart gets that spot because of their mobile app which does a pretty good job of telling me what is in stock at my local store. My other top choice is a small (three store) chain in Western Washington called Central Market [1]. (If the name seems familiar, it may be from season 10 of "Top Chef" which was set in Seattle. Top Chef normally sends…
New Seasons Markets in Oregon and Washington are doing all these steps as well.
Re: I had to build a web scraper to buy groceries
#75Few months ago someone had posted open source scraper that got 1k+ votes (or maybe not) - it was used by NYTimes for realtime alerts. Someone remind me please.
You're talking about Huginn, right?
Re: I had to build a web scraper to buy groceries
#76Re: I had to build a web scraper to buy groceries
#77Of all the grocery companies in my area, only Walmart has it right: Choose an available pick up time, then start online shopping. All the others let you fill your cart, then at checkout tell you there are no slots available until the end of time. Waste of time, energy, and hope. F U, Target, Safeway, Albertsons, and Kroger.
That's insane. All the online grocery shops here in the UK work that way (pick slot first then pick items).
There are still no slots at all available anywhere for either delivery or collection though.
Re: I had to build a web scraper to buy groceries
#78I’ve done a few of those scrapers and mail to notify projects in the past, but I’ve recently switched from using email to a telegram bot, it’s pretty great. Very easy to implement and doesn’t spam your inbox.
Re: I had to build a web scraper to buy groceries
#79Few months ago someone had posted open source scraper that got 1k+ votes (or maybe not) - it was used by NYTimes for realtime alerts. Someone remind me please.
You're talking about Huginn, right? https://news.ycombinator.com/item?id=21772610
To clarify, Huginn is a framework/app server for scrapers and doing rest calls etc based on them
Re: I had to build a web scraper to buy groceries
#80Hehe, I had to do the same a week ago. The only difference is Slack’ing instead of mailing and also cart prepopulation because those delivery slots are taken out almost instantly in my city. This is in the EU btw, not specifying the country or chain to, well, to be able to survive longer.