Three fundamental tricks for developers writing distributed systems
pedro.herokuapp.com
Three fundamental tricks for developers writing distributed systems
1–10 of 20 posts
Re: Three fundamental tricks for developers writing distributed systems
#2The one thing that concerns me about using the database as a queue is that MVCC doesn't really lend itself to writing threading primitives like locks. I'm curious how one would go about writing a queue in an MVCC architecture--off the top of my head, I guess you could have a job assignments table to link processors and jobs, make the job FK unique and interpret forced rollbacks as indicating that another thread grabbed the job before you did. Then again, if your queue is only running idempotent functions it wouldn't matter if you had more than one thread doing the same work, it would just be a waste of time.
Re: Three fundamental tricks for developers writing distributed systems
#3The one thing that concerns me about using the database as a queue is that MVCC doesn't really lend itself to writing threading primitives like locks. I'm curious how one would go about writing a queue in an MVCC architecture--off the top of my head, I guess you could have a job assignments table to link processors and jobs, make the job FK unique and interpret forced rollbacks as indicating that another thread grabb…
I believe one can make a stored procedure that implements an atomic enqueue or dequeue operation in a transaction. Still, such systems seem to end up polling, or having triggers that launch non-database activities such as scripts or network activity.
Note that I'm not endorsing this scheme.
Re: Three fundamental tricks for developers writing distributed systems
#4Isn't this the same as a message queue? Why would you want to rewrite this using a database? Also point/trick 2 seems unnecessary if you are using 3 (idempotent jobs). By queuing job ids you now have a consistency dependency between your message queue and database.
Re: Three fundamental tricks for developers writing distributed systems
#5Isn't this the same as a message queue? Why would you want to rewrite this using a database? Also point/trick 2 seems unnecessary if you are using 3 (idempotent jobs). By queuing job ids you now have a consistency dependency between your message queue and database.
When using a non database-based queue you'll have to find another mechanism to make sure your operation is still atomic.
In other words you can end up in a situation where a record is inserted but the job to work on it is not enqueued, or worse - that an insert fails but the job to work on it is enqueued.
Point 2 is still necessary despite idempotency imo: lets say some value is updated to "a" and then to "b", enqueuing two jobs. If the request to update "b" runs before "a" then your receiver will end up with the wrong value. Same if the initial request to update "a" fails.
Re: Three fundamental tricks for developers writing distributed systems
#6Many times the idea behind distributed systems is to avoid single point of failure. by using Database for communication, you are essentially creating another single point of failure in the form of database (unless database is running on some highly reliable elaborate master-master setup). However you can use systems like zookeeper to get similar functionality and to facilitate communication.
Re: Three fundamental tricks for developers writing distributed systems
#7Isn't this the same as a message queue? Why would you want to rewrite this using a database? Also point/trick 2 seems unnecessary if you are using 3 (idempotent jobs). By queuing job ids you now have a consistency dependency between your message queue and database.
When using a message queue you're forced/expected to flush messages quickly, this is often not the case.
Re: Three fundamental tricks for developers writing distributed systems
#8Isn't this the same as a message queue? Why would you want to rewrite this using a database? Also point/trick 2 seems unnecessary if you are using 3 (idempotent jobs). By queuing job ids you now have a consistency dependency between your message queue and database.
When using a non database-based queue you'll have to find another mechanism to make sure your operation is still atomic. In other words you can end up in a situation where a record is inserted but the job to work on it is not enqueued, or worse - that an insert fails but the job to work on it is enqueued. Point 2 is still necessary despite idempotency imo: lets say some value is updated to "a" and then to "b", enqueu…
Perhaps I wasn't clear. What I meant was that if you enqueue the job directly and the job is idempotent, using approach 2 will introduce dependencies like the ones you pointed out.
Re: Three fundamental tricks for developers writing distributed systems
#9Isn't this the same as a message queue? Why would you want to rewrite this using a database? Also point/trick 2 seems unnecessary if you are using 3 (idempotent jobs). By queuing job ids you now have a consistency dependency between your message queue and database.
When using a message queue you're forced/expected to flush messages quickly, this is often not the case.
There are lots of message queues that overcome this. A good example is Apache Kafka (http://incubator.apache.org/kafka/)
Re: Three fundamental tricks for developers writing distributed systems
#10Earlier quoted context omitted.
When using a message queue you're forced/expected to flush messages quickly, this is often not the case.
There are lots of message queues that overcome this. A good example is Apache Kafka ( http://incubator.apache.org/kafka/ )
Actually I was considering mentioning Kafka when I wrote that comment and contrasting it with RabbitMQ.