Varlink – IPC to replace D-Bus gradually in systemd
1–10 of 273 posts
Re: Varlink – IPC to replace D-Bus gradually in systemd
#2That looks interesting, I especially appreciate the intentional design decision for documentation and schemas. While it didn't work quite well with dbus Ime it is still a good idea to make APIs more accessible.
Re: Varlink – IPC to replace D-Bus gradually in systemd
#3So Varlink requires a proper specification of message types, but then uses god damn JSON to serialize messages? Why would you do that?
Apparently, we don't have enough wasted cycles in modern software, just add a bunch more in a fundamental IPC infrastructure.
Re: Varlink – IPC to replace D-Bus gradually in systemd
#4So Varlink requires a proper specification of message types, but then uses god damn JSON to serialize messages? Why would you do that? Apparently, we don't have enough wasted cycles in modern software, just add a bunch more in a fundamental IPC infrastructure.
Lets take it a level up. I'm still not sure why we even need a message bus in there in the first place. The whole Linux boot, init, systemd pile is a Rube Goldberg machine that is very difficult to understand at the moment. The only reason I suspect most people aren't complaining is that our abstraction level is currently a lot higher (docker / kubernetes etc) and machines are mostly ephemeral cattle and few people go near it.
As for JSON, please don't even start me on that. It is one of the worst serialisation decisions we ever made as a society. Poorly defined and unreliable primitive types, terrible schema support and expensive to parse. In a kernel, it does not belong! Give it a few months and varlink will be YAML over the top of that.
Re: Varlink – IPC to replace D-Bus gradually in systemd
#5So Varlink requires a proper specification of message types, but then uses god damn JSON to serialize messages? Why would you do that? Apparently, we don't have enough wasted cycles in modern software, just add a bunch more in a fundamental IPC infrastructure.
Unfortunately I concur.
There are a lot of problems with D-Bus, but that it wasn't using JSON wasn't one of them.
The tooling for D-Bus is terrible. It has composite types but they are very primitive, e.g. you can't name fields in a structure without extensions (there's like three different extensions for this.) A lot of the code generators can't actually handle real-world schemas.
Now I understand that D-Bus itself is also a problem (as in, the protocol design beyond serialization) but varlink looks like a very solid step backwards. In any single individual use case JSON serialization is unlikely to be a huge bottleneck, but in practice if this will be used for all kinds of things it will add up.
I really wish the Linux desktop would coalesce around something like Cap'n'proto for serialization.
P.S.: It'd probably be possible to shoehorn in multiple formats but I think that's a mistake too. Something like capnp will have a strong impedance mismatch with JSON; mixing formats with different degrees of self-describability is unwise. But then formats like MsgPack and BSON which are direct JSON analogues miss out on the best benefits of using binary serialization formats, which makes them almost the worst of both worlds, and you'd probably be better off with using a highly-optimized JSON library like simdjson.
Re: Varlink – IPC to replace D-Bus gradually in systemd
#6So Varlink requires a proper specification of message types, but then uses god damn JSON to serialize messages? Why would you do that? Apparently, we don't have enough wasted cycles in modern software, just add a bunch more in a fundamental IPC infrastructure.
Lets take it a level up. I'm still not sure why we even need a message bus in there in the first place. The whole Linux boot, init, systemd pile is a Rube Goldberg machine that is very difficult to understand at the moment. The only reason I suspect most people aren't complaining is that our abstraction level is currently a lot higher (docker / kubernetes etc) and machines are mostly ephemeral cattle and few people g…
> I'm still not sure why we even need a message bus in there in the first place.
Because traditional POSIX IPC mechanisms are absolute unworkable dogshit.
> It is one of the worst serialisation decisions we ever made as a society.
There isn't really any alternative. It's either JSON or "JSON but in binary". (Like CBOR.) Anything else is not interoperable.
Re: Varlink – IPC to replace D-Bus gradually in systemd
#7So Varlink requires a proper specification of message types, but then uses god damn JSON to serialize messages? Why would you do that? Apparently, we don't have enough wasted cycles in modern software, just add a bunch more in a fundamental IPC infrastructure.
Lets take it a level up. I'm still not sure why we even need a message bus in there in the first place. The whole Linux boot, init, systemd pile is a Rube Goldberg machine that is very difficult to understand at the moment. The only reason I suspect most people aren't complaining is that our abstraction level is currently a lot higher (docker / kubernetes etc) and machines are mostly ephemeral cattle and few people g…
I think it doesn't have to be a message bus per-se, that design decision is mostly just because it's convenient. Even D-Bus can actually be used without the bus, if you really want to.
D-Bus is just a bus so it can solve a bunch of different IPC problems at the same time. e.g. D-Bus can handle ACLs/permissions on RPC methods, D-Bus can handle multiple producers and multiple consumers, and so on. I think ultimately all that's really needed is IPC, and having a unified bus is just to allow for some additional use cases that are harder if you're only using UNIX domain sockets directly.
If there are going to be systemd components that have IPC, then I'd argue they should probably use something standard rather than something bespoke. It's good to not re-invent the wheel.
Not that I think Varlink is any better. It seems like at best it's probably a lateral move. I hope this does not go forward.
Re: Varlink – IPC to replace D-Bus gradually in systemd
#8Earlier quoted context omitted.
Lets take it a level up. I'm still not sure why we even need a message bus in there in the first place. The whole Linux boot, init, systemd pile is a Rube Goldberg machine that is very difficult to understand at the moment. The only reason I suspect most people aren't complaining is that our abstraction level is currently a lot higher (docker / kubernetes etc) and machines are mostly ephemeral cattle and few people g…
> I'm still not sure why we even need a message bus in there in the first place. Because traditional POSIX IPC mechanisms are absolute unworkable dogshit. > It is one of the worst serialisation decisions we ever made as a society. There isn't really any alternative. It's either JSON or "JSON but in binary". (Like CBOR.) Anything else is not interoperable.
This is a quite frankly ridiculous point. Most of that garb came from the HPC people who built loads of stuff on top of it in the first place. It's absolutely fine for this sort of stuff. It's sending the odd little thing here and there, not on a complex HPC cluster.
As for JSON, are you really that short sighted that it's the only method of encoding something? Is "oh well it doesn't fit the primitive types, so just shove it in a string and add another layer of parsing" acceptable? Hell no.
Re: Varlink – IPC to replace D-Bus gradually in systemd
#9So Varlink requires a proper specification of message types, but then uses god damn JSON to serialize messages? Why would you do that? Apparently, we don't have enough wasted cycles in modern software, just add a bunch more in a fundamental IPC infrastructure.
There was a Varlink talk[0] a few days ago at All Systems Go, in that talk, Lennart mentioned that JSON is unfortunate (primarily due to no 64 bit ints) but it has the surprising benefit of being able to understand the bus messages when using `strace` for debugging.
[0]: https://www.youtube.com/watch?v=emaVH2oWs2Y&list=PLWYdJViL9E...
Re: Varlink – IPC to replace D-Bus gradually in systemd
#10Earlier quoted context omitted.
Lets take it a level up. I'm still not sure why we even need a message bus in there in the first place. The whole Linux boot, init, systemd pile is a Rube Goldberg machine that is very difficult to understand at the moment. The only reason I suspect most people aren't complaining is that our abstraction level is currently a lot higher (docker / kubernetes etc) and machines are mostly ephemeral cattle and few people g…
I think it doesn't have to be a message bus per-se, that design decision is mostly just because it's convenient. Even D-Bus can actually be used without the bus, if you really want to. D-Bus is just a bus so it can solve a bunch of different IPC problems at the same time. e.g. D-Bus can handle ACLs/permissions on RPC methods, D-Bus can handle multiple producers and multiple consumers, and so on. I think ultimately al…
> If there are going to be systemd components that have IPC, then I'd argue they should probably use something standard rather than something bespoke. It's good to not re-invent the wheel.
This is my point.
My favourite DBus situation a number of years ago was a CentOS 7 box that reboot stopped working on with a cryptic DBus error that no one has ever seen before. I had to sync it, power cycle the node from the ILO card and cross my fingers.
I really don't give a shit about this. I just wanted to run my jobs on the node, not turn into a sysadmin due to someone else's dubious architectural decisions.