41–50 of 69 posts
Implementing proposed approach is not easy without hurting maintenance, scalability and readability. At Google, there is a dedicated framework for writing distributed systems that implements proposals from the document.
> The call to hello.Greet looks like a regular method call
That’s a departure from how components interact in boq — an internal and widely used production platform that has _some_ of the features from the paper. There component interfaces _are_ RPC interfaces (e.g., Stubby / gRPC + protocol buffers), and interaction between them is possible exclusively through the component interfaces. Hence it’s very explicit at the call site that an RPC is being made (which could happen to execute locally with all the standard RPC functionality — context and deadline propagation, etc.).
RPCs looking like regular method calls sound a bit scary (easy to miss in code reviews); I wonder if enforced naming conventions + IDE + code review tool support would be enough.
Edit: it seems to require to pass a context object, so the readers won't confuse it with a local call (from https://serviceweaver.dev/):
sum, err := adder.Add(ctx, 1, 2)
Also, the paper claims that most benefits come from a non-versioned serialization format:
> Most of the performance benefits of our prototype come from its use of a custom serialization format designed for non-versioned data exchange [...]
However, I don’t understand why local RPC calls have to serialize protocol buffer messages — can’t they already pass them as-is to the local handler?
(disclaimer: a googler, no internal knowledge on ServiceWeaver)
Mobile code and "agent systems" were very fashionable 20 years ago. Java introduced built-in RMI with automatic stub downloading. In 1998 Sun published https://en.wikipedia.org/wiki/Jini that was an extension of this idea. Several higher level frameworks emerged (JavaSpaces among the most prolific). Reading the paper two thoughts come to mind: - "What's old is new again" - "those who do not learn from history are doo…
> Java RMI use a programming model similar to ours but suffered from a number of technical and organizational issues [58] and don’t fully address C1-C5 either
Apart from that, it looks like Java RMI allowed remote objects returning other remote objects, rather than only immutable values. With that you could abuse it by making a call to one java.rmi.Remote object, getting another java.rmi.Remote object in response, then passing it around, and then finding a totally different subsystem suddenly make RPCs (however, such abuse probably would be easy to spot in a code review, as it requires a modification to the remote object interface).
---
The authors also acknowledge that it doesn’t solve the distributed computing challenges:
> our proposal does not solve fundamental challenges of distributed systems [53, 68, 76]. Application developers still need to be aware that components may fail or experience high latency
I think at least in terms of latencies their platform can occasionally inject latencies into some percentage of the tasks, then verifying if any alerts fire, if there is a fear the components become dependent on a certain deployment shape (within a cluster).
Bold claims... Performance is evaluated against a single example implementation (section 6.1) and 9x improvement was achieved when co-locating into a single process. To be fair this seems a reasonable thing to do if you can get the application to be efficient enough to allow it - but there may be very good reasons not to do it in many applications.
Mobile code and "agent systems" were very fashionable 20 years ago. Java introduced built-in RMI with automatic stub downloading. In 1998 Sun published https://en.wikipedia.org/wiki/Jini that was an extension of this idea. Several higher level frameworks emerged (JavaSpaces among the most prolific). Reading the paper two thoughts come to mind: - "What's old is new again" - "those who do not learn from history are doo…
The paper unfortunately hides that in reality you have to pass a context object in your RPC calls, hence there is no ambiguity whether you are calling a potentially remote object.
It's in the example on the project home page: https://serviceweaver.dev/
// The "RPC" handler
func (adder) Add(_ context.Context, x, y int) (int, error) {
return x + y, nil
}
// The call-site
var adder Adder = ... // See documentation
sum, err := adder.Add(ctx, 1, 2)- Making remote calls seem like they are local resulted in poor design decisions, the benefit of SOAP/REST was that people considered what the interface a useful service should be.
- Why not flip it and look to move groups of microservices onto the same machine, updating how the app communicate. - if component boundaries are fine grained, the combinations of local/remote services relative to each other increases, along with the testing burden; just because the system hides remote deploying, it still should be tested for.
- Incorporating this with storage, eg dynamic shard rebalancing would be super cool
Bold claims... Performance is evaluated against a single example implementation (section 6.1) and 9x improvement was achieved when co-locating into a single process. To be fair this seems a reasonable thing to do if you can get the application to be efficient enough to allow it - but there may be very good reasons not to do it in many applications.
Network I/O is almost always the slowest part of any application regardless of the protocol and data encoding mechanism. It seems to reason that if you can co-locate your calls into a single process, you'd gain at least 9x.
This intuition is what encouraged us to run with SQLite in production back when most developers didn't think it was a good idea.
Looks interesting. > The call to hello.Greet looks like a regular method call That’s a departure from how components interact in boq — an internal and widely used production platform that has _some_ of the features from the paper. There component interfaces _are_ RPC interfaces (e.g., Stubby / gRPC + protocol buffers), and interaction between them is possible exclusively through the component interfaces. Hence it’s v…
I didn't read the paper in enough detail to know the answer to this, but mightn't this enable different implementation languages for different components? In my experience, it's difficult to accomplish reliably that without using a language-agnostic serialization format (like proto).
Even if that's the goal, it seems like a handler could determine whether it could elide the serialization depending on the implementation details of the components.
Mobile code and "agent systems" were very fashionable 20 years ago. Java introduced built-in RMI with automatic stub downloading. In 1998 Sun published https://en.wikipedia.org/wiki/Jini that was an extension of this idea. Several higher level frameworks emerged (JavaSpaces among the most prolific). Reading the paper two thoughts come to mind: - "What's old is new again" - "those who do not learn from history are doo…
Regarding the comparison with RMI, the authors did mention it: > Java RMI use a programming model similar to ours but suffered from a number of technical and organizational issues [58] and don’t fully address C1-C5 either Apart from that, it looks like Java RMI allowed remote objects returning other remote objects, rather than only immutable values. With that you could abuse it by making a call to one java.rmi.Remote…
Actually this is a very powerful concept as it allows one to achieve high level of reuse. Jini (https://jan.newmarch.name/java/jini/tutorial/Jini.html) made mobile objects the core idea of its architecture.
How powerful the approach is one can see when looking for example at Jini concept of a Lease and LeaseRevenewalService:
a server program can register an object (client side implementation of a service) in ServiceRegistrar (to make it discoverable and downloadable). Registration is lease based so the server has to renew it periodically. But it can be delegated to a LeaseRenewalService that can do it on its behalf so that the server can go to sleep (ie. not use any server machine resources).
All of the above happens without any party a-priori knowledge about any code that needs to be present at use site - code is downloaded automatically on-demand - the only thing common to client and service is a Java interface.