Rails, callbacks, workers, and the race you never expected to lose • Hacker News Board

Re: Rails, callbacks, workers, and the race you never expected to lose

#21

Post by drumdance » Tue, Aug 21, 2012, 3:05 PM UTC

kennystone wrote: post #18

This is so much more easily solved by just sending the JSON document of the user in the message to the worker. A database look is not necessary, a transactional record lookup based on ID is overkill.

Maybe. We send different messages based on a user's subscription level which is two tables away ( user.account.subscription). I suppose we could dump that into the json object too, but IMO that just adds complexity.

Re: Rails, callbacks, workers, and the race you never expected to lose

#22

Post by strictfp » Tue, Aug 21, 2012, 3:33 PM UTC

This can be solved by using a transactional messaging system. The message will then only enter the queue when all datasources have committed. This also has the additional benefit that a tx rollback after the entity save will cancel the message, and not crash the message handler.

Re: Rails, callbacks, workers, and the race you never expected to lose

#23

Post by urbanautomaton » Tue, Aug 21, 2012, 3:44 PM UTC

latortuga wrote: post #16

urbanautomaton wrote: post #12

Earlier quoted context omitted.

He's not using the model to render the view, though, he's using it to queue up a future asynchronous request to the relevant mail "controller".

Sure, you're right, but only technically. Having an extra collaborator in the model is asking for trouble - there's nothing about a User that requires it to know anything about emailing. It violates all kinds of design principles. Putting your logic for "I should email the User when they sign up" into the User model is using "design by related words". In traditional Rails design, the controller is exactly the right p…

Ah, technically correct - the best kind of correct. :-)

Anyway, I agree with pretty much everything you say. As I mentioned elsewhere in this thread, we've gone down a pub-sub event system for handling our generic post-action tasks. The controller announces a :create_user event, and any interested listeners can respond appropriately. And yes, one of the biggest benefits of this has been the ability to test that event-handling logic in isolation; stubbing and mocking collaborators, and generally having the sort of testing fun you're normally not supposed to have with Rails.

Re: Rails, callbacks, workers, and the race you never expected to lose

#24

Post by andyl » Tue, Aug 21, 2012, 5:42 PM UTC

I've started using the DCI technique outlined in Jim Gay's book 'Clean Ruby'. To me Jim's technique is somewhat similar to the Hexagonal Rails approach describe in another comment.

The DCI technique is to use the model as a persistence layer, and organize business logic into modules organized by use cases. With this technique my system has become much easier to understand and test.

Its great to see people experimenting with architecture styles like DCI and HexRails.

Re: Rails, callbacks, workers, and the race you never expected to lose

#25

Post by kennystone » Tue, Aug 21, 2012, 5:48 PM UTC

drumdance wrote: post #21

kennystone wrote: post #18

This is so much more easily solved by just sending the JSON document of the user in the message to the worker. A database look is not necessary, a transactional record lookup based on ID is overkill.

Maybe. We send different messages based on a user's subscription level which is two tables away ( user.account.subscription). I suppose we could dump that into the json object too, but IMO that just adds complexity.

Looking up state and dealing with race conditions is way more complex than fire and forget, imo.

Re: Rails, callbacks, workers, and the race you never expected to lose

#26

Post by pardner » Tue, Aug 21, 2012, 5:50 PM UTC

kennystone wrote: post #18

This is so much more easily solved by just sending the JSON document of the user in the message to the worker. A database look is not necessary, a transactional record lookup based on ID is overkill.

We do something similar, but a little simpler. Each object in our database has a short guid. (In our case, the guids have a type-identifier so by inspection we can tell if a guid is for a user, or text message, or whatever.) We pass the guid to the worker.

Re: Rails, callbacks, workers, and the race you never expected to lose

#27

Post by atomical » Tue, Aug 21, 2012, 6:25 PM UTC

I think this problem would be solved with after_create. Is there a reason not to use this?

Re: Rails, callbacks, workers, and the race you never expected to lose

#28

Post by ruckusing » Wed, Aug 22, 2012, 1:53 AM UTC

I encountered this very issue a few years ago, basically your queue picking up jobs before the database commits. I solved this by actually queuing up my jobs to a thread-local queue and then firing the whole spool at the end of the request (in an around_filter, so you fire the spool after the yield to the action). This means that you actually fire your jobs after the request has done its whole database commit. This can all be done without having to leak abstractions and make the model layer know about the presence or lack of the fake queue.

Let me know if anyone is interested and I can share some code.