Waltz: A Distributed Write-Ahead Log
wecode.wepay.com
Waltz: A Distributed Write-Ahead Log
1–10 of 46 posts
Re: Waltz: A Distributed Write-Ahead Log
#2This design can be extended to support interactive transactions.
Here's how:
1. Assign each transaction a log position. For strict serializability instead of just serializability, make this the highest log position ever acknowledged successfully to a client. This can be batched for throughput.
2. Have each client record their read set and write set, which includes the objects/ranges the transaction read and the writes / operations to be performed.
3. Have clients persist this R/W set into the log, or send directly to the lock server if it is also the master assigning log positions. Again, use batching for throughput.
4. Have your lock server either as a part of the master processing assigning log positions, or have it tail the log separately. The lock server will receive batches of transactions, take locks in the log's defined order, then commit another entry to the log with the commit/abort decision for each.
5. Respond to the client with the commit / abort decision.
To make this easier to program you'll probably want to include a read-your-writes cache on the client.
You can also scale out this log by having clients commit their operations to multiple logs. The only thing that needs to be serialized is the locking and commit / abort decision making. These other logs can be sharded by key range or just chosen randomly, as long as the commit / abort decision log includes which other log the data itself was committed to.
FoundationDB works roughly like this. The terminology is different (resolver instead of lock server, R/W set is conflict ranges, log position is version number, etc) but this is basically how it works.
Re: Waltz: A Distributed Write-Ahead Log
#3This design can be extended to support interactive transactions. Here's how: 1. Assign each transaction a log position. For strict serializability instead of just serializability, make this the highest log position ever acknowledged successfully to a client. This can be batched for throughput. 2. Have each client record their read set and write set, which includes the objects/ranges the transaction read and the write…
Persisting executed transactions to a log with their read/write sets, and then determining if they actually commit also starts to sound like http://web.eecs.umich.edu/~michjc/eecs584/Papers/cidr11_hyde... to me
Re: Waltz: A Distributed Write-Ahead Log
#4This design can be extended to support interactive transactions. Here's how: 1. Assign each transaction a log position. For strict serializability instead of just serializability, make this the highest log position ever acknowledged successfully to a client. This can be batched for throughput. 2. Have each client record their read set and write set, which includes the objects/ranges the transaction read and the write…
Persisting executed transactions to a log with their read/write sets, and then determining if they actually commit also starts to sound like http://web.eecs.umich.edu/~michjc/eecs584/Papers/cidr11_hyde... to me
Yes, it is more similar to Hyder than FoundationDB if you persist the R/W sets to the log itself. FoundationDB gets around this by only keeping the lock table in memory. When any process in the transaction pipeline fails, the master (which holds the version number) advances the version far enough into the future that any in-progress transaction would fail because it is too old.
Re: Waltz: A Distributed Write-Ahead Log
#5How does this compare to Kafka?
Re: Waltz: A Distributed Write-Ahead Log
#6How does this compare to Kafka?
The first paragraph on that page covers that at a high level.
> Waltz is similar to existing log systems like Kafka in that it accepts/persists/propagates transaction data produced/consumed by many services. However, unlike other systems, Waltz provides a machinery that facilitates a serializable consistency in distributed applications. It detects conflicting transactions before they are committed to the log.
Re: Waltz: A Distributed Write-Ahead Log
#7Last I heard from some friends in YC when considering a position there, Wepay was handling less than 10^6 payments per day. Is that still the case and is something with such low requirements a good replacement for Kafka in the wild?
Re: Waltz: A Distributed Write-Ahead Log
#8How does this compare to Kafka?
The first paragraph on that page covers that at a high level. > Waltz is similar to existing log systems like Kafka in that it accepts/persists/propagates transaction data produced/consumed by many services. However, unlike other systems, Waltz provides a machinery that facilitates a serializable consistency in distributed applications. It detects conflicting transactions before they are committed to the log.
> serializable consistency in distributed applications.
Kakfa supports transactions not sure about serializable though.
Re: Waltz: A Distributed Write-Ahead Log
#9Here's a dumb question about log-structured systems like this: does this system work nicely with backfills? Suppose you start logging events with Waltz and you want to migrate an existing system's data into the same log. Or something goes wrong and oncall needs to manually insert old events. Does Waltz have capabilities to backfill events into the historical log or reassign transaction IDs?
This might not be needed if this is strictly used for FIFO event consumption, but I guess I was thinking of trying to make a system like this support time-sliced queries.
Re: Waltz: A Distributed Write-Ahead Log
#10> This works well, but the drawback is that a service has to write to two separate storage systems, a database and Kafka. We still need check-and-repair.
what about kafka connect ?