21–30 of 57 posts
Of course, the network of queues isn't the only thing that makes the system possible, but it closely resembles processing in a distributed system; a system of workers.
It's clear from Fowler's discussion of "mechanical sympathy" that he didn't understand the memory hierarchy. I found it pretty shocking that someone in his position could not know this stuff. He really needs to read Ulrich Drepper's "What every programmer should know about memory" ( http://www.akkadia.org/drepper/cpumemory.pdf ). It's such a fundamental topic that everyone ought to at least know the basics of it. Peo…
In particular, since each input event has to be journaled and replicated (which both involve I/O) before it can be processed, there is a potentially large and unpredictable delay for each event.
Another issue is that this architecture assumes that 1 CPU core is enough to run all business logic processors, and that 1 machine's memory is enough to hold your state. If your processing is something CPU-intensive or your state is large, you'll hit a scaling wall that requires you to manually shard your input across multiple machines.
For people who want to implement this approach in C++, FastFlow is the library to use (and it's crazy fast): http://calvados.di.unipi.it/dokuwiki/doku.php?id=ffnamespace...
Earlier quoted context omitted.
The Disruptor components are not part of a single threaded process, as far as I understand "Also these three tasks are relatively independent, all of them need to be done before the Business Logic Processor works on a message, but they can done in any order. So unlike with the Business Logic Processor, where each trade changes the market for subsequent trades, there is a natural fit for concurrency." I think that the…
The BLP is single-threaded, and the component that modifies the application's current state. It is my understanding that "Events" in event-sourcing are immutable and that state is just a memoized computation over the total journal of events -- so the layer of atomiticity that is provided is at the event level (as the BLP only processes one event at a time.)
Hmm.
The "network of queues" architecture reminds me of Prof. Matt Welsh's PhD thesis titled SEDA: Staged Event Driven Architecture. It has benefits of both the concurrency offered by event driven and parallelism offered by threaded architectures. http://www.eecs.harvard.edu/~mdw/proj/seda/ Of course, the network of queues isn't the only thing that makes the system possible, but it closely resembles processing in a distri…
In general, this reminds me a lot of architectures designed for embedded systems today (which are how software was designed for PC's in the early days).
The huge up side is performance the huge down side is that it completely ignores the significance of what a relational database offers to the company as a whole.
We need to be looking at ways to make SQL databases faster, not at ways to avoid its use.
I didn't see a place to add comments on his article page. In general, this reminds me a lot of architectures designed for embedded systems today (which are how software was designed for PC's in the early days). The huge up side is performance the huge down side is that it completely ignores the significance of what a relational database offers to the company as a whole. We need to be looking at ways to make SQL datab…
The upside is performance, because that's what is considered important in this case. The downside, which you've neglected to mention is difficulty of maintenance due to decreased comprehensibility of the system as a whole.
A relational database may not have that much value to the company as a whole.
By all means look at ways to make SQL DBMSes quicker. Lots of clever people have spent decades doing just that. I'm sure they're not finished yet. [incidentally Mohan et al, were using sequential writes for their [undo - or redo? I don't recall which] logs back when CPU speeds where measured in double-digit MHz - DB2 and all that. While its been a while since I had cause to look at the code inside any DBMS, I suspect the same is true today.]
But talking about SQL here is just a hammer in search of a thumb. Pick the tool for the job.
[in summary, I'm more than happy to keep building high-performance low-latency embedded systems in ways that might make an applications programmer weep, but I'm quite glad that the folks who take care of my company's payroll are running industrial-strength transactional systems]