71–80 of 157 posts
A notable example of a large-scale app built with a very similar architecture is ATproto/Bluesky[1]. "ATProto for Distributed Systems Engineers" describes how updates from the users end up in their own small databases (called PDS) and then a replicated log. What we traditionally think of as an API server (called a view server in ATProto) is simply one among the many materializations of this log. I personally find thi…
This is lovely and I'm itching to try it. One question:
We have a use case where a location gets cut off completely from the internet at large. In that case, it makes sense for the local hardware (typically Android and/or iOS tablets or equivalent) to take over as a log owner: even though you're cut off, if you're willing to swallow the risk (and hence cost) of offline payments, you should be able to create orders, fulfill them, pay for them, close them out, send tickets to the kitchen to cook the food or to the warehouse to fetch the tractor, etc.
Does restate include something that covers that use-case? In the noodling/daydreaming a colleague and I have done, we ended up with something very close to restate (I imagined just using Kafka), except that additionally many operations would have a CRDT nature: eg. you should _always_ be allowed to add a payment to an order, because presumably a real-life payment happened.
I've also noodled with the idea of logs whose canonical ownership can be transferred. That covers cases where you start offline and then reconnect, but doesn't work so well for transactions that start out connected (and thus owned in the datacenter) and need to continue offline.
One could also imagine ensuring that > n/2 consensus members are always located inside the restaurant/hardware store/etc., so if you go offline, you can still proceed. It might even be possible to recognize disconnection and then take one full vote to further subdivide that pool of consensus members so if one dies it doesn't halt progress. This feels like it would be getting very tricksy…
> Having a single place (the one log) that forces a linear history of events as the ground truth and owns the decision of who can add to that ground truth, means we don’t have to coordinate much any more. Well, yes, but then you've backed into CAP again because you only have one log.
couldn't the log be synchronously replicated to multiple servers to increase A?
How would you compare this to the actor model or to temporal?
The Virtual Objects in Restate are much like actors. They are somewhat inspired by Orleans [1], and you could call them virtual stateful actors. They blend with the durable execution for processing messages with multiple durable steps.
Regarding temporal, check also this question: https://news.ycombinator.com/item?id=42815318
I learned about distributed incrementally -monotonic logs back at the late 90s, with many other ways to do guaranteed transactional database actions. And I'm quite certain these must have been invented in the 50s or 60s, as these are the problems that early business computer users had: banking software. These are the techniques that were buried in legacy COBOL routines, and needed to be slowly replaced by robust Java core services.
I'm sure the Restate designers will have learned terribly useful insights in how to translate these basic principles into a working system with the complexities of today's hardware/software ecosystem.
Yet it makes me wonder if young programmers are only being taught the "build fast-break things" mentality and there are no longer SW engineers able to insert these guarantees into their systems from the beginning, by standing on the shoulders of the ancients that invented our discipline, so that their lore is actually used in practice? Or am I just missing something new in the article that describes some novel twist?
Earlier quoted context omitted.
Calling a WAL a ledger, why? Ledger sounds fancier but why would it be a ledger in this case?
I believe "ledger" implies commutative property (order does not matter).
I have seen it used to mean WAL before, so I am taking this with a dose of skepticism.
Earlier quoted context omitted.
couldn't the log be synchronously replicated to multiple servers to increase A?
In case of mutation either the replicas will be out of sync (so no C), or you'll need to synchronously mutate the replicas (so bad A due to latency).
Earlier quoted context omitted.
In case of mutation either the replicas will be out of sync (so no C), or you'll need to synchronously mutate the replicas (so bad A due to latency).
delaying writes by a factor longer than the max clock skew of the cluster is a pretty common strategy. It is what Google Spanner does.
The max clock skew also has to pick between A and C, in underestimates you lose serializability, in overestimates you pay in write latency.
Excuse me for sounding rough, but - isn't this reinventing comp-sci, one step at a time? I learned about distributed incrementally -monotonic logs back at the late 90s, with many other ways to do guaranteed transactional database actions. And I'm quite certain these must have been invented in the 50s or 60s, as these are the problems that early business computer users had: banking software. These are the techniques t…
Here's a follow-up thought: to what extent did the grey-beards let us juniors down by steering us down a different path? A few instances:
DB creators knew about replicated logs, but we got given DBs, not replicated log products.
The Java creators knew about immutability: "I would use an immutable whenever I can." [James Gosling, 1] but it was years later when someone else provided us with pcollections/javaslang/vavr. And they're still far from widespread, and nowhere near the standard library.
Brendan Eich supposedly wanted to put Scheme into browsers, but his superiors had him make JS instead.
What other stuff have we been missing out on?
[1] https://www.artima.com/articles/james-gosling-on-java-may-20...