Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is the cheapest, easiest way to host a cronjob in 2022?

news.ycombinator.com

Post reply on HN
Print view View original

111–120 of 336 posts

Re: Ask HN: What is the cheapest, easiest way to host a cronjob in 2022?

#111

Post by MuffinFlavored »

SeriousM wrote: post #72
MuffinFlavored wrote: post #53

Earlier quoted context omitted.

Eh. I tried to do Chromium/Puppeteer based scraping this way. Building a Dockerfile took ages due to the low compute. (Rust was a non-starter). I also had (foolishly) only bought the Pi with 2GB instead of 8GB so RAM was an issue. Disk was super slow. I'm not sure how viable this is, especially with how hard it is currently to source a Pi, let alone its computation/memory constraints.

You can use the "browserless" docker service which contains a headless chrome browser in a docker container. It also supports puppeteer and playwright connect api. Works flawless! I use it in combination with n8n. All on a raspberry pi4b (yes, I got one recently)

> You can use the "browserless" docker service which contains a headless chrome browser in a docker container.

A lot of websites can detect the IPs from this and block it, basically almostly like a CAPTCHA.

I had other needs like Postgres, etc.

Scraping data is one thing, actually doing anything with it is another. I quickly hit the limits of a $50 RaspberryPi 4 or whatever they're going for on Amazon these days with the gouging, etc.

Re: Ask HN: What is the cheapest, easiest way to host a cronjob in 2022?

#117

Post by alexeldeib »

johnmoberg wrote: post #29

You should check out Modal! Does much more than just cronjobs, but super easy and cheap: https://modal.com/docs/guide/cron . They have an example with a simple web scraper as well: https://modal.com/docs/guide/web-scraper From their pricing example: > You schedule a job to run once an hour for 15 seconds. This job uses 50% of one CPU core and 256 MiB of memory. This job will cost $0.016/day for the CPU, and $0.001/da…

This is very cool. Love the pricing model.

Re: Ask HN: What is the cheapest, easiest way to host a cronjob in 2022?

#118

Post by Waterluvian »

I set up a Raspberry Pi on my local network because every year I end up with dozens of little use cases for a Web addressable, always-on machine where I DO NOT need redundancy, uptime guarantees, backups, CDNs, high bandwidth needs, etc.

I want to offload this, but any time I search for an option, they're overcomplicated or overpriced. Maybe there just isn't a market for it?

...maybe I don't want to offload this. The Pi and SD card was like $35, the power is basically zero, and the Internet is basically zero.

Re: Ask HN: What is the cheapest, easiest way to host a cronjob in 2022?

#119

Post by gwn7 »

MuffinFlavored wrote: post #53
gwn7 wrote: post #23

Earlier quoted context omitted.

Yes yes yes. I came to post the exact same thing then saw this =)

Eh. I tried to do Chromium/Puppeteer based scraping this way. Building a Dockerfile took ages due to the low compute. (Rust was a non-starter). I also had (foolishly) only bought the Pi with 2GB instead of 8GB so RAM was an issue. Disk was super slow. I'm not sure how viable this is, especially with how hard it is currently to source a Pi, let alone its computation/memory constraints.

You're mentioning the worst possible scenario.

First of all the answer recommends to use either a Pi or a VPS. If a Pi doesn't cut it for you, just switch to a VPS with sufficient specs for your requirement. Problem solved, now it's viable.

Besides, a significant part of the web can be scraped without resorting to a heavyweight browser such as Chromium. It should always be the last resort. Even if you have to evaluate Javascript (in case of SPAs), there are much cheaper solutions than Puppeteer (JSDOM being an example) which can get the job done most of the time.

As to Docker, I fail to see why you would need Docker for this kind of job, unless you don't know how to do it without Docker.

When there is no portability requirement, the costs of Docker easily surpass its benefits.

...

So no, it's not that what's being recommended in the answer is not viable. You're doing it wrong. Either you're using a Pi when you need to use a VPS or you are introducing unnecessary layers.

Post reply on HN