Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Varlink – IPC to replace D-Bus gradually in systemd

varlink.org

Post reply on HN
Print view View original

111–120 of 273 posts

Re: Varlink – IPC to replace D-Bus gradually in systemd

#111

Post by Spivak »

userbinator wrote: post #100
theamk wrote: post #38

Earlier quoted context omitted.

Sigh. Varlink is designed for IPC, so the expected data rate is maybe hundreds of messages per second in the worst case. JSON can be parsed at 3 Gigabytes/second [0]. Even unoptimized stdlib parsers in scripting languages get 50 MB/sec. That's more than enough, standard library support is much more important for a project like this. And if there is ever a reason to send tar archive or video data, there is always "upg…

It doesn't matter what the rate is. Of all the things they could do, they decided to choose one of the worst possible formats for IPC data[1], maybe to pander to the web crowd. [1] Maybe XML would be a close competitor.

It's the lingua-franca serialization format that has bindings to every programming language already, doesn't require agreeing on the payload layout beforehand, is beyond overkill for the task being asked of it, and is human debuggable over the wire.

I'm begging you to read the format it's replacing. https://dbus.freedesktop.org/doc/dbus-specification.html Ctrl+f Type System. JSON is beautiful, elegant, and minimal by comparison. yyyyuua(yv) It's like the printf message protocol but worse.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#112

Post by quotemstr »

cryptonector wrote: post #107
dfox wrote: post #66

Earlier quoted context omitted.

ASN.1 BER/DER is more or less the same thing as CBOR. The perceived complexity of ASN.1 comes from the schema language and specifications written in the convoluted telco/ITU-T style (and well, the 80's type system that has ~8 times two different types for “human readable string”).

That "convoluted telco/ITU-T style" yields amazingly high quality specifications. I'll take X.68x / X.69x any day over most Internet RFCs (and I've written a number of Internet RFCs). The ITU-T puts a great deal of effort into its specs, or at least the ASN.1 working group did. ASN.1 is not that complicated. Pity that fools who thought ASN.1 was complicated re-invented the wheel quite poorly (Protocol Buffers I'm loo…

For our sins, our industry is doomed to suffer under unbearable weight of endless reinvented wheels. Of course it would have been better to stick with ASN.1. Of course we didn't, because inexperience and hubris. We'll never learn.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#113

Post by cryptonector »

RandomThoughts3 wrote: post #84
dfox wrote: post #66

Earlier quoted context omitted.

ASN.1 BER/DER is more or less the same thing as CBOR. The perceived complexity of ASN.1 comes from the schema language and specifications written in the convoluted telco/ITU-T style (and well, the 80's type system that has ~8 times two different types for “human readable string”).

> The perceived complexity of ASN.1 comes from the schema language and specifications written in the convoluted telco/ITU-T style (and well, the 80's type system that has ~8 times two different types for “human readable string”). I can’t resist pointing that it’s basically a longer way of saying quite complicated and not very efficient.

> I can’t resist pointing that it’s basically a longer way of saying quite complicated and not very efficient.

That's very wrong. ASN.1 is complicated because it's quite complete by comparison to other syntaxes, but it's absolutely not inefficient unless you mean BER/DER/CER, but those are just _some_ of the encoding rules available for use with ASN.1.

To give just one example of "complicated", ASN.1 lets you specify default values for optional members (fields) of SEQUENCEs and SETs (structures), whereas Protocol Buffers and XDR (to give some examples) only let you specify optional fields but not default values.

Another example of "complicated" is that ASN.1 has extensibility rules because the whole "oh TLV encodings are inherently extensible" thing turned out to be a Bad Idea (tm) when people decided that TLV encodings were unnecessarily inefficient (true!) so they designed efficient, non-TLV encodings. Well guess what: Protocol Buffers suffers from extensibility issues that ASN.1 does not, and that is a serious problem.

Basically, with a subset of ASN.1 you can do everything that you can do with MSFT RPC's IDL, with XDR, with Protocol Buffers, etc. But if you stick to a simple subset of ASN.1, or to any of those other IDLs, then you end up having to write _normative_ natural language text (typically English) in specifications to cover all the things not stated in the IDL part of the spec. The problem with that is that it's easy to miss things or get them wrong, or to be ambiguous. ASN.1 in its full flower of "complexity" (all of X.680 plus all of X.681, X.682, and X.683) lets you express much more of your protocols in a _formal_ language.

I maintain an ASN.1 compiler. I've implemented parts of X.681, X.682, and X.683 so that I could have the compiler generate code for the sorts of typed holes you see in PKI -all the extensions, all the SANs, and so on- so that the programmer can do much less of the work of having to invoke a codec for each of those extensions.

A lot of the complexity in ASN.1 is optional, but it's very much worth at least knowing about it. Certainly it's worth not repeating mistakes of the past. Protocol Buffers is infuriating. Not only is PB a TLV encoding (why? probably because "extensibility is easy with TLV!!1!, but that's not quite true), but the IDL requires manual assignment of tag values, which makes uses of the PB IDL very ugly. ASN.1 originally also had the manual assignment of tags problem, but eventually ASN.1 was extended to not require that anymore.

Cavalier attitudes like "ASN.1 is too complicated" lead to bad results.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#115

Post by cryptonector »

NekkoDroid wrote: post #27
jmclnx wrote: post #20

Does this mean Lunux will need both Dbus and varlink if using systemd ? I ask because I believe Firefox uses Dbus. https://www.phoronix.com/news/Systemd-Varlink-D-Bus-Future

systemd has been using varlink for a while now along side dbus, its not something just now being introduced. They don't have a problem living side-by-side.

It's more code, more bloat, more complexity, and more attack surface area.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#116

Post by cryptonector »

quotemstr wrote: post #112
cryptonector wrote: post #107

Earlier quoted context omitted.

That "convoluted telco/ITU-T style" yields amazingly high quality specifications. I'll take X.68x / X.69x any day over most Internet RFCs (and I've written a number of Internet RFCs). The ITU-T puts a great deal of effort into its specs, or at least the ASN.1 working group did. ASN.1 is not that complicated. Pity that fools who thought ASN.1 was complicated re-invented the wheel quite poorly (Protocol Buffers I'm loo…

For our sins, our industry is doomed to suffer under unbearable weight of endless reinvented wheels. Of course it would have been better to stick with ASN.1. Of course we didn't, because inexperience and hubris. We'll never learn.

It sure seems that way. Sad. It's not just hubris nor inexperience -- it's cognitive load. It's often easier to wing something that later grows a lot than it is to go find a suitable technology that already exists.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#117

Post by quotemstr »

Spivak wrote: post #111
userbinator wrote: post #100

Earlier quoted context omitted.

It doesn't matter what the rate is. Of all the things they could do, they decided to choose one of the worst possible formats for IPC data[1], maybe to pander to the web crowd. [1] Maybe XML would be a close competitor.

It's the lingua-franca serialization format that has bindings to every programming language already, doesn't require agreeing on the payload layout beforehand, is beyond overkill for the task being asked of it, and is human debuggable over the wire. I'm begging you to read the format it's replacing. https://dbus.freedesktop.org/doc/dbus-specification.html Ctrl+f Type System. JSON is beautiful, elegant, and minimal by…

JSON isn't simple. It's simplistic. It's too simple. By refusing to acknowledge the complexity of the real world at the protocol layer, varlink shifts the burden of dealing with the complexity from one transport layer to N ad hoc application layers, making us all worse off.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#118

Post by KerrAvon »

nly wrote: post #70
arianvanp wrote: post #25

Earlier quoted context omitted.

CBOR is the obvious choice here. I've suggested it before and the systemd folks didn't seem completely opposed to it. Also because cbor parser is already in the dependencies due to FIDO2 disk unlocking

It's not the data format that is important, but the tooling, code generation and binding quality.

Data format is absolutely important for operating system IPC. JSON is raw text. Messages occupy more RAM than a binary format, which increases your working set, and it requires at least some nontrivial parsing. This ultimately limits the number of messages you can send per second.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#119

Post by eviks »

quotemstr wrote: post #108

Christ almighty, I have seldom seen a technical proposal as bad as this one. I want off Mr. Poettering's wild ride. No 64 bit integers? No file descriptor transport? No message sequence numbers? Just cut it out already. Dbus doesn't do something? Extend dbus. Don't make some other random and worse things.

> Extend dbus.

How would you address the issue of dbus unavailability in early boot listed in the presentation via extension?

Re: Varlink – IPC to replace D-Bus gradually in systemd

#120

Post by KerrAvon »

XorNot wrote: post #87
theamk wrote: post #39

Earlier quoted context omitted.

What is the data rate (messages, bytes) that you expect for this "D-Bus replacement protocol"? Which fraction of CPU will JSON ser/des will take to justify using CBOR?

I mean conversely every second week on HN someone complains about "inefficient modern development practices" and this is the sort of minor thing which would be trotted out as an example. So I'd argue it's not an unreasonable question (although I lean closer in: better types then JSON is the problem, since a decent text serialization format is always going to be needed for debugging and development).

This sounds like it wants to be a fundamental IPC facility in Linux, which seems like a major thing.

Any binary format can be dumped to a text when viewed for debugging. That’s really a nonissue. Have the tools dump to JSON if you like.

Post reply on HN