Resque with Redis To Go

#1

Post by waratuman » Wed, Jul 28, 2010, 3:53 PM UTC

Resque with Redis To Go
blog.redistogo.com

Re: Resque with Redis To Go

#2

Post by aaronblohowiak » Wed, Jul 28, 2010, 4:04 PM UTC

Use the best tool for the job. For queueing, that's probably 0mq or ActiveMQ or RabbitMQ, depending on your needs.

Re: Resque with Redis To Go

#3

Post by klochner » Wed, Jul 28, 2010, 4:07 PM UTC

Great idea - I don't believe there was previously a Redis solution for Heroku users.

I'm not seeing an obvious benefit for people self-hosting though.

Re: Resque with Redis To Go

#4

Post by rramsden » Wed, Jul 28, 2010, 4:19 PM UTC

If you're looking for something a little simpler to process jobs using mongodb check out mongo_queue http://github.com/skiz/mongo_queue

Re: Resque with Redis To Go

#5

Post by waratuman » Wed, Jul 28, 2010, 4:25 PM UTC

aaronblohowiak wrote: post #2

Use the best tool for the job. For queueing, that's probably 0mq or ActiveMQ or RabbitMQ, depending on your needs.

These are primarily messaging systems. I've considered using them before. Nothing, however, has come close to letting me do introspection like Resque does, and the best part is that I don't need to do any work to get it.

Re: Resque with Redis To Go

#6

Post by callmeed » Wed, Jul 28, 2010, 4:31 PM UTC

Looks pretty awesome. I'd love to use it on an upcoming project.

2 questions:

1. Do jobs (in app/jobs/whatever.rb) have full access to your Rails models (so you can pass it an ID and it can do whatever)?

2. How does the rake task/worker best work in a production environment (other than Heroku)? Use monit to keep the worker(s) up?

Thanks

Re: Resque with Redis To Go

#7

Post by delano » Wed, Jul 28, 2010, 4:34 PM UTC

Great stuff!

Also, for the extremely anal, uri-redis is aware of the database number:

    require 'uri-redis'
    conf = URI.parse 'redis://localhost:6379/1'
    conf.host      # => localhost
    conf.port      # => 6379
    conf.db        # => 1

Re: Resque with Redis To Go

#8

Post by malomalo » Wed, Jul 28, 2010, 4:36 PM UTC

callmeed wrote: post #6

Looks pretty awesome. I'd love to use it on an upcoming project. 2 questions: 1. Do jobs (in app/jobs/whatever.rb) have full access to your Rails models (so you can pass it an ID and it can do whatever)? 2. How does the rake task/worker best work in a production environment (other than Heroku)? Use monit to keep the worker(s) up? Thanks

1. Yes

2. Just use the "rake resque:work" task and I recommend using something like upstart that works awesome, but I have also done it with monit and both work great

Re: Resque with Redis To Go

#9

Post by aaronblohowiak » Wed, Jul 28, 2010, 4:48 PM UTC

waratuman wrote: post #5

aaronblohowiak wrote: post #2

Use the best tool for the job. For queueing, that's probably 0mq or ActiveMQ or RabbitMQ, depending on your needs.

These are primarily messaging systems. I've considered using them before. Nothing, however, has come close to letting me do introspection like Resque does, and the best part is that I don't need to do any work to get it.

They are message queues. What makes introspection a critical requirement?

Re: Resque with Redis To Go

#10

Post by waratuman » Wed, Jul 28, 2010, 4:55 PM UTC

aaronblohowiak wrote: post #9

waratuman wrote: post #5

Earlier quoted context omitted.

These are primarily messaging systems. I've considered using them before. Nothing, however, has come close to letting me do introspection like Resque does, and the best part is that I don't need to do any work to get it.

They are message queues. What makes introspection a critical requirement?

For me it was because I was unable to access the log files if a job failed since I was on Heroku. Also it gives me a general idea of how the system behaves; in one second I can tell if the system is normal. I can tell if I need to bump some workers up for a certain queue. Basically it saves me from creating my own 'dashboard'.