HireFire - The Heroku Worker Manager
github.com
HireFire - The Heroku Worker Manager
1–10 of 20 posts
Re: HireFire - The Heroku Worker Manager
#2Looks extremely cool. Can't wait to kick the tires.
Re: HireFire - The Heroku Worker Manager
#3Very handy. Was looking at using http://www.simpleworker.com/ as an alternative to the uncost effective Heroku DJ's but hopefully HireFire will sway me back to using Heroku.
Re: HireFire - The Heroku Worker Manager
#4Very handy. Was looking at using http://www.simpleworker.com/ as an alternative to the uncost effective Heroku DJ's but hopefully HireFire will sway me back to using Heroku.
Hey. I was actually also using SimpleWorker prior to HireFire, but I find that setting it up is a bit hard / takes long since you have to set isolated code / credentials / etc to their servers in order to process background jobs. Also, the jobs aren't guaranteed to run immediately after they are queued, may take up to 10 minutes on the cheapest plan. When deployed to Heroku it's easy since Heroku will just use the same compiled slug it uses for Dyno's, no additional code/configuration required, it just works. I also found that on SimpleWorker a job that took only 3 seconds on Heroku, would run for 30-90 seconds on SimpleWorker (no idea why).
That said. There are some advantages when using SimpleWorker such as the web interface, scheduling recurring jobs, and other useful options. Especially when you aren't running on Heroku, and memory is an issue if you have a small VPS. In that case, offloading background jobs to SimpleWorker is ideal. But I find that HireFire will make Heroku Workers cheaper than SimpleWorker due to the fact that jobs appear to take (a lot) longer to finish on SimpleWorker than on Heroku (probably due to the start up time? I'm not sure why).
Re: HireFire - The Heroku Worker Manager
#5For the low-traffic scenario there's also an older, much simpler solution: https://github.com/mtravers/cheepnis
Re: HireFire - The Heroku Worker Manager
#6[deleted]
Re: HireFire - The Heroku Worker Manager
#7I was looking for something like this months ago. Thanks for building, and I'm looking forward to (at least trying to) contribute!
Re: HireFire - The Heroku Worker Manager
#8Very handy. Was looking at using http://www.simpleworker.com/ as an alternative to the uncost effective Heroku DJ's but hopefully HireFire will sway me back to using Heroku.
Hey. I was actually also using SimpleWorker prior to HireFire, but I find that setting it up is a bit hard / takes long since you have to set isolated code / credentials / etc to their servers in order to process background jobs. Also, the jobs aren't guaranteed to run immediately after they are queued, may take up to 10 minutes on the cheapest plan. When deployed to Heroku it's easy since Heroku will just use the sa…
Couldn't agree more. The setup time of SimpleWorker is alot ( especially when dealing with specific gems ) and the fact that for DB work you have to create a separate non concurrent connection to the DB per job makes those times you talk about a reality. Ok, you could use the 'expensive jobs' but Heroku's DJ's win hands down in all tests I've done. Plus who doesn't like the handle_asynchronously :function.
Re: HireFire - The Heroku Worker Manager
#9Earlier quoted context omitted.
Hey. I was actually also using SimpleWorker prior to HireFire, but I find that setting it up is a bit hard / takes long since you have to set isolated code / credentials / etc to their servers in order to process background jobs. Also, the jobs aren't guaranteed to run immediately after they are queued, may take up to 10 minutes on the cheapest plan. When deployed to Heroku it's easy since Heroku will just use the sa…
Couldn't agree more. The setup time of SimpleWorker is alot ( especially when dealing with specific gems ) and the fact that for DB work you have to create a separate non concurrent connection to the DB per job makes those times you talk about a reality. Ok, you could use the 'expensive jobs' but Heroku's DJ's win hands down in all tests I've done. Plus who doesn't like the handle_asynchronously :function.
Exactly. :) The DB connection is also quite frustrating depending on the location of the DB. MongoDB with MongoHQ or MongoMachine is fine since it's external already. But otherwise you have to know how to configure for example your MySQL PostgreSQL instances to allow incoming connections from external locations, and also becomes potentially insecure depending on your sysadmin skills.
I find Heroku + DJ + HireFire so much simpler, either using that handle_asynchronous method, or just called delay.my_method(*args) is brilliant. No need to think about isolated code or credentials due to the environment variables that are already set on Heroku.
I should look in to integrating Navvy and Resque as well.
Re: HireFire - The Heroku Worker Manager
#10Nice work, glad to see heroku API gems getting some love.
I wrote a similar gem back in January and have had good luck with it: https://github.com/freerobby/middle_management
It's fully configurable via environment variables, so you can make changes without redeploying. It also lets you specify a minimum number of workers (even if there are zero jobs) so that you don't have to wait for workers to spin up if you don't want to.