The LMAX Architecture - 100K TPS at Less than 1ms Latency
martinfowler.com
The LMAX Architecture - 100K TPS at Less than 1ms Latency
1–10 of 57 posts
Re: The LMAX Architecture - 100K TPS at Less than 1ms Latency
#2This is great stuff, thanks very much for posting this.
Re: The LMAX Architecture - 100K TPS at Less than 1ms Latency
#3As usual, it depends on what you define as a "transaction".
If you don't need ACID, I can probably jerry-rig something in C that'll give you hundreds of thousands of "transactions" per second. It may have problems with retrieving the data, but hey, look at that sucker go!
OK. Time to be more reasonable.
1. Relaxed Guarantees
There's no actual durability in the conventional spinning-rust sense. Instead you're hoping that a fleet of identical instances all receive the same event objects in the same order. If they don't, you're going to have mysterious problems when some of the systems get out of sync.
According to fowler, business transactions are broken into events which are processed individually. This essentially means that some transactions aren't atomic. LMAX can credit A and fail to debit B because the bits are done independently.
Consistency is palmed off to the input queues.
2. Smart data structures
They profiled the code and found places where they could swap out standard Java collections for their own problem-specific variants.
What they refer to as "Disruptors" are smart ring buffers; or as they are sometimes called, "Queues". I realise that this is not as cool-sounding as "Disruptor" (they should have called the "Business Logic Processor" the "Phaser"!), but it seems to more or less describe the interface, which is that things go in a given order and come out in approximately that order.
Actually, it's possible I've misunderstood how that structure works. It might also be that the system is skipping over elements that aren't yet ready, in which case this looks more like an active priority queue, interface-wise.
3. Conclusion
Impressive work from the LMAX team. But let's remember to keep stuff in perspective. It has always been the case that ACID exacts a high toll on performance. To the extent that you relaxed it you could always go faster.
Too often we in our industry see the shiny big performance number and forget that it isn't free. Like everything in life there is an opportunity cost. Choose carefully.
Re: The LMAX Architecture - 100K TPS at Less than 1ms Latency
#4For people who want to implement this approach in C++, FastFlow is the library to use (and it's crazy fast): http://calvados.di.unipi.it/dokuwiki/doku.php?id=ffnamespace...
Re: The LMAX Architecture - 100K TPS at Less than 1ms Latency
#5As usual, it depends on what you define as a "transaction". If you don't need ACID, I can probably jerry-rig something in C that'll give you hundreds of thousands of "transactions" per second. It may have problems with retrieving the data, but hey, look at that sucker go! OK. Time to be more reasonable. 1. Relaxed Guarantees There's no actual durability in the conventional spinning-rust sense. Instead you're hoping t…
In regards to your comments:
1. relaxed guarantees, "As a consequence the LMAX team puts a lot of attention into ensuring the input events are fully valid before doing any mutation of the in-memory persistent state.". It looks like they deal with inconsistency at the business logic level instead of delegating to the persistence store. Probably, they don't have very complex transactional logic.
2. Smart data structures, here http://disruptor.googlecode.com/files/Disruptor-1.0.pdf you have a deeper technical view of the Disruptor and the rationale behind "mechanical sympathy".
Re: The LMAX Architecture - 100K TPS at Less than 1ms Latency
#6As usual, it depends on what you define as a "transaction". If you don't need ACID, I can probably jerry-rig something in C that'll give you hundreds of thousands of "transactions" per second. It may have problems with retrieving the data, but hey, look at that sucker go! OK. Time to be more reasonable. 1. Relaxed Guarantees There's no actual durability in the conventional spinning-rust sense. Instead you're hoping t…
As far as I understand three of the key features are:
1. Reduced queue contention: queues are typically implemented with a list, e.g. linked list, this introduces contention (queues spend a lot of time empty or very full) for the head and tail of the queue which are often the same dummy node. The ring buffer removes this contention.
2. Machine Sympathy vis a vis cache striding and ensuring concurrent threads are not invalidating each others level 1/2 cache.
3. Pre-allocation of queue data structures to ensure GC is not a factor.
Personally I think the LMAX team have done well in advancing the state of the art in what is often a key component in event driven, high throughput low latency systems such as those used in banks for trading, exchanges and market data.
Re: The LMAX Architecture - 100K TPS at Less than 1ms Latency
#7As usual, it depends on what you define as a "transaction". If you don't need ACID, I can probably jerry-rig something in C that'll give you hundreds of thousands of "transactions" per second. It may have problems with retrieving the data, but hey, look at that sucker go! OK. Time to be more reasonable. 1. Relaxed Guarantees There's no actual durability in the conventional spinning-rust sense. Instead you're hoping t…
In regards to your comments: 1. relaxed guarantees, "As a consequence the LMAX team puts a lot of attention into ensuring the input events are fully valid before doing any mutation of the in-memory persistent state.". It looks like they deal with inconsistency at the business logic level instead of delegating to the persistence store. Probably, they don't have very complex transactional logic. 2. Smart data structure…
> It looks like they deal with inconsistency at the business logic level instead of delegating to the persistence store.
Actually it looks the other way around. As I read it, the "Input Disruptor" does the validation.
Do the three components all run in the same thread, or are they on different JVMs?
Re: The LMAX Architecture - 100K TPS at Less than 1ms Latency
#8Earlier quoted context omitted.
In regards to your comments: 1. relaxed guarantees, "As a consequence the LMAX team puts a lot of attention into ensuring the input events are fully valid before doing any mutation of the in-memory persistent state.". It looks like they deal with inconsistency at the business logic level instead of delegating to the persistence store. Probably, they don't have very complex transactional logic. 2. Smart data structure…
> It looks like they deal with inconsistency at the business logic level instead of delegating to the persistence store. Actually it looks the other way around. As I read it, the "Input Disruptor" does the validation. Do the three components all run in the same thread, or are they on different JVMs?
The Disruptor components are not part of a single threaded process, as far as I understand
"Also these three tasks are relatively independent, all of them need to be done before the Business Logic Processor works on a message, but they can done in any order. So unlike with the Business Logic Processor, where each trade changes the market for subsequent trades, there is a natural fit for concurrency."
I think that the Disruptor can run on the save JVM as the business logic, also to avoid remoting-related bottlenecks.
Re: The LMAX Architecture - 100K TPS at Less than 1ms Latency
#9Earlier quoted context omitted.
> It looks like they deal with inconsistency at the business logic level instead of delegating to the persistence store. Actually it looks the other way around. As I read it, the "Input Disruptor" does the validation. Do the three components all run in the same thread, or are they on different JVMs?
The Disruptor components are not part of a single threaded process, as far as I understand "Also these three tasks are relatively independent, all of them need to be done before the Business Logic Processor works on a message, but they can done in any order. So unlike with the Business Logic Processor, where each trade changes the market for subsequent trades, there is a natural fit for concurrency." I think that the…
The hand off between producer and consumer can be configured to be both synchronous (single threaded) as well as asynchronous (one producer thread, many consumers threads).
Re: The LMAX Architecture - 100K TPS at Less than 1ms Latency
#10How do they save their data from that All-in-memory-single-thread-JVM nonsense into persistent storage? ^_^
What is that All-in-memory-single-thread-JVM crap segfaults?