Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Varlink – IPC to replace D-Bus gradually in systemd

varlink.org

Post reply on HN
Print view View original

51–60 of 273 posts

Re: Varlink – IPC to replace D-Bus gradually in systemd

#51

Post by brookst »

anotherhue wrote: post #47
deivid wrote: post #9

Earlier quoted context omitted.

There was a Varlink talk[0] a few days ago at All Systems Go, in that talk, Lennart mentioned that JSON is unfortunate (primarily due to no 64 bit ints) but it has the surprising benefit of being able to understand the bus messages when using `strace` for debugging. [0]: https://www.youtube.com/watch?v=emaVH2oWs2Y&list=PLWYdJViL9E...

With this logic (oft repeated) we should be sending TCP as JSON. { sourcePort: 443, destinationPort: 12345,...} Debugability is important but the answer is to build debugging tools, not to lobotomise the protocol for the vanishingly tiny fraction of packets that are ultimately subject to debugging.

I love your TCP over JSON idea. Maybe we can implement layer 2 that way as well, with CSMA/CD: {startTime:2024-09-29T13:56:21.06551, duration:00:00:00.0024, packet: …}

Re: Varlink – IPC to replace D-Bus gradually in systemd

#52

Post by ptx »

Setting the serialization format issue aside, why is the D-Bus broker an issue?

Why was it designed that way in the first place if all we need (supposedly) is a socket and a simple request/response protocol? If it wasn't needed, why does D-Bus have it? And if it is needed, what will replace it when systemd switches to Varlink?

Re: Varlink – IPC to replace D-Bus gradually in systemd

#53

Post by ptx »

jsnell wrote: post #12
newpavlov wrote: post #3

So Varlink requires a proper specification of message types, but then uses god damn JSON to serialize messages? Why would you do that? Apparently, we don't have enough wasted cycles in modern software, just add a bunch more in a fundamental IPC infrastructure.

I've done similar things in the past (json over udp-to-localhost, with a schema description used to generate the code for both parsing and generating on each end). It's a totally reasonable point in the design space, and I literally never saw a reason to revisit the decision to use json for that system. You'd do that because everything under the sun knows how to generate and parse json. Performance looks like a total…

> Performance looks like a total non-issue for this use case.

The presentation brings up performance as one of the problems with the D-Bus though, so it seems like it is an issue.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#54

Post by Xylakant »

6SixTy wrote: post #49
RandomThoughts3 wrote: post #46

Earlier quoted context omitted.

I’m having trouble following what you mean. Yes, you can certainly send a whole SQLite database using any binary communication protocol as a way to exchange data today. You can even compress it before if you feel like it. It will not make for a good ipc format because, well, it’s not designed to be one but it would most definitely work. What’s your point?

My point is, what's stopping me from creating yet another IPC standard?

Nothing. It might not see adoption, but you can create as many IPC standards as you desire.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#55

Post by jsnell »

ptx wrote: post #53
jsnell wrote: post #12

Earlier quoted context omitted.

I've done similar things in the past (json over udp-to-localhost, with a schema description used to generate the code for both parsing and generating on each end). It's a totally reasonable point in the design space, and I literally never saw a reason to revisit the decision to use json for that system. You'd do that because everything under the sun knows how to generate and parse json. Performance looks like a total…

> Performance looks like a total non-issue for this use case. The presentation brings up performance as one of the problems with the D-Bus though, so it seems like it is an issue.

Fair. What I meant is that the performance delta between JSON and any other interchange data format should be a non-issue in this context. I would be quite surprised to hear that any performance problems systemd has with D-Bus was with serialization/de-serialization.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#56

Post by IshKebab »

jsnell wrote: post #55
ptx wrote: post #53

Earlier quoted context omitted.

> Performance looks like a total non-issue for this use case. The presentation brings up performance as one of the problems with the D-Bus though, so it seems like it is an issue.

Fair. What I meant is that the performance delta between JSON and any other interchange data format should be a non-issue in this context. I would be quite surprised to hear that any performance problems systemd has with D-Bus was with serialization/de-serialization.

Yeah... it's easy to say JSON won't be an issue. It's difficult to switch from JSON when it turns out you were wrong.

https://raphlinus.github.io/xi/2020/06/27/xi-retrospective.h...

Given how central this is going to be why would you skip the tiny bit of extra work needed to use a binary format?

Re: Varlink – IPC to replace D-Bus gradually in systemd

#57

Post by IshKebab »

capitainenemo wrote: post #37
gmokki wrote: post #17

Earlier quoted context omitted.

I do not understand where the 64bit integers not working with JSON comes from. JSON the format had no limit on integer size. And all Java JSON libraries I know can handle arbitrary prevsion integers (BigInt) and 32/64bit int/long types when serializing and deserializing. Quick googling shows that also JavaScript has proper support for 64bit integers with their BigInt type, and it can be used to deserialize incoming d…

The number type in JSON is 64 bit float, limiting integers without loss of precision to 2⁵³-1. BigInt is a new concept and not technically supported. So whether it works in your random library of choice is probably a potshoot. "Use within JSON: Using JSON.stringify() with any BigInt value will raise a TypeError, as BigInt values aren't serialized in JSON by default. " https://developer.mozilla.org/en-US/docs/Web/Java…

That's not true. The number type in JSON is an arbitrary sized number. You're thinking of JavaScript which is not the same thing.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#58

Post by AshamedCaptain »

altfredd wrote: post #48
AshamedCaptain wrote: post #42

Earlier quoted context omitted.

Same reason Android (or PalmOS) didn't decide to use any of the existing IPC mechanisms, and why then Luna/webOS also didn't decide to use Binder even though they had experience with it and reinvent something else. That urge that forces every developer to reinvent RPC every other week.

Not really. Binder solves a real-world problem — priority inversion. It is also the only IPC on Linux that allows to perform a blocking call from process A to process B (and have process B synchronously call back to process A if needed). D-Bus, Varlink and other "IPC solutions" on top of sockets can not do those things. Android uses synchronous Binder calls for 90% of it's application APIs, so there is clearly a real…

Binder was not even designed for Linux. The primary reason it is used is that (at the time) they wanted to abstract from Linux. "They" here is not Android/Google.

Binder also specifies a message format -- even if not fully, since the kernel is going to peek into your message for things like FDs, binder objects, etc. Or your userspace is going to need to "special treat" these fields somehow.

It competes in the same space as D-Bus no doubt. If the story of Android had been different, with the companies spinning off some years later, it may have very well ended up using Luna RPC (https://www.webosbrew.org/pages/luna-service-bus.html ) (which is practically indistinguishable from D-Bus, bar the JSON) or anything else, really.

> D-Bus, Varlink and other "IPC solutions" on top of sockets can not do those things.

WTF? Sure you can do on top of even pipes. Even XDR could...

"Synchronous" is a very loose word here if you mean to be interrupted in the middle of a RPC call, anyway.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#59

Post by IshKebab »

hi-v-rocknroll wrote: post #28
newpavlov wrote: post #3

So Varlink requires a proper specification of message types, but then uses god damn JSON to serialize messages? Why would you do that? Apparently, we don't have enough wasted cycles in modern software, just add a bunch more in a fundamental IPC infrastructure.

Sigh. Cap'n Proto already exists. Reinventing the wheel yet again because NIH.

CapnProto is kind of awful though. I had to use it for several years and the API is extremely unfriendly.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#60

Post by akira2501 »

arianvanp wrote: post #25
jchw wrote: post #5

Earlier quoted context omitted.

Unfortunately I concur. There are a lot of problems with D-Bus, but that it wasn't using JSON wasn't one of them. The tooling for D-Bus is terrible. It has composite types but they are very primitive, e.g. you can't name fields in a structure without extensions (there's like three different extensions for this.) A lot of the code generators can't actually handle real-world schemas. Now I understand that D-Bus itself…

CBOR is the obvious choice here. I've suggested it before and the systemd folks didn't seem completely opposed to it. Also because cbor parser is already in the dependencies due to FIDO2 disk unlocking

I genuinely dislike CBOR. Formats which require me to calculate the length of the message before sending it lead to bad library design which often leads to easily compromised code. Add in an "indefinite length" option and you've got potentially unbounded client memory usage to watch out for. As if that wasn't enough you get extensible tags so the meaning of any message is entirely dependent on the context it was sent in.

It gives you a lot of decent protocol wire design and then flatly ignores everything we've learned about these types of designs in the last 3 decades. Be on the lookout for client libraries to slowly add in all of these checks as the vulnerabilities are discovered in them.

Post reply on HN