Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Varlink – IPC to replace D-Bus gradually in systemd

varlink.org

Post reply on HN
Print view View original

231–240 of 273 posts

Re: Varlink – IPC to replace D-Bus gradually in systemd

#231

Post by orbisvicis »

tssva wrote: post #220
orbisvicis wrote: post #198

Earlier quoted context omitted.

> Primarily append-based > Seekable Text logs are append-based and seekable as well. > Support for in-line Forward Secure Sealing I once typed an SSH password in the username field, and the only way to erase that was to erase all the logs. So this has some significant downsides. Also, I am tired of waiting minutes for a journalctl query to load.

> I once typed an SSH password in the username field, and the only way to erase that was to erase all the logs. So this has some significant downsides. I hope this was a personal system. Changing logs in this manner would have almost certainly led to your dismissal anywhere I ever worked. This anecdote just re-enforces the need for Forward Secure Sealing.

No, I just left it there.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#233

Post by eternityforest »

newpavlov wrote: post #3

So Varlink requires a proper specification of message types, but then uses god damn JSON to serialize messages? Why would you do that? Apparently, we don't have enough wasted cycles in modern software, just add a bunch more in a fundamental IPC infrastructure.

How many messages is this actually gonna handle? If it's tens per second, should you maybe be using something else?

JSON is so prevalent I'm pretty sure it's a lot of people's mental model of what non relational structured data looks like, if it's not just a file.

It's nice from a dev perspective to always work directly with the same high level concepts that you use when thinking about this stuff.

Other data formats are often advertised in terms of JSON, either subsets or supersets, or equivalent representations.

I'd use something else for high performance stuff.... But for everything else it's pretty awesome.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#234

Post by xarope »

Someone explain to me please, how type-safe and JSON can even exist in the same sentence? JSON has so many edge cases for handling numbers, floats, bools etc. Why would you not use gRPC or even something like CapnProto if you want to maintain that type-safety-ness?

Re: Varlink – IPC to replace D-Bus gradually in systemd

#235

Post by eternityforest »

thayne wrote: post #216
poettering wrote: post #125

Earlier quoted context omitted.

The marshalling cost for JSON is negligible. Yes, it might be a bit slower than GVariant for example, but only by some fractional linear factor. And on small messages (which D-Bus currently always is, due to message size constraints enforced by broker) the difference is impossible to measure. To a point it really doesn't matter, in particular as JSON parsers have been ridiculously well optimized in this world. What d…

What if varlink supported both JSON and a subset of cbor with the "cbor data follows" tag at the beginning (so the server can determine if it is json or cbor based on the beginning of the message)? It would add a little complexity to the server, but then clients can choose if they want to use a human readable format that has more available libraries or a binary format. As for strace, tooling could probably be added t…

But why though? Is this really a performance critical bus?

Re: Varlink – IPC to replace D-Bus gradually in systemd

#236

Post by RandomThoughts3 »

cryptonector wrote: post #206
RandomThoughts3 wrote: post #156

Earlier quoted context omitted.

> Uh, no, structs, records, whatever you want to call them It's plenty clear from discussion context that OP is talking about C struct but yes, replace C with any languages which suit you. It will still be part of the language semantic and not an IPC specification. The point is you can't generally use memory layout as an IPC protocol because you generally have no guarantee that it will be the same for all architectur…

If it's IPC, it's the same architecture (mostly; typically there's at most 3 local architectures). The receiver can always make right. If there's hidden remoting going on, the proxies can make things right.

> The receiver can always make right.

Certainly but that’s hardly structs anymore. You are implicitly defining a binary format which is aligned on the sender memory layout then.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#237

Post by jchw »

cryptonector wrote: post #121
jchw wrote: post #29

Earlier quoted context omitted.

C structs are a terrible serialization format, since they are not a serialization format at all. Nothing guarantees that you will get consistent struct behavior on the same machine, but also, it only really solves the problem for C. For everything else, you have to duplicate the C structure exactly, including however it may vary per architecture (e.g. due to alignment.) And OK fine. It's not that bad, most C ABIs are…

BTW, Lustre's RPC system's serialization is very much based on C structs. It's receiver-makes-right to deal with endianness, for example. It's a pain, but it's also fast.

Making an RPC serialization system that is zero-overhead i.e. can use the same format on the wire as it does on disk is not a terrible idea. Capnp is the serialization format that I've been suggesting as a potential candidate and it is basically just taking the idea of C structs, dumping it into it's own schema language, and adding the bare minimum to get some protobuf-like semantics.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#238

Post by musicnarcoman »

eternityforest wrote: post #235
thayne wrote: post #216

Earlier quoted context omitted.

What if varlink supported both JSON and a subset of cbor with the "cbor data follows" tag at the beginning (so the server can determine if it is json or cbor based on the beginning of the message)? It would add a little complexity to the server, but then clients can choose if they want to use a human readable format that has more available libraries or a binary format. As for strace, tooling could probably be added t…

But why though? Is this really a performance critical bus?

Yes. I run shared desktop login server clusters for students, DBUS is a bottle neck.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#239

Post by guappa »

xorcist wrote: post #222
guappa wrote: post #153

Earlier quoted context omitted.

Do you know what strace is? It's a command that prints the system calls (done via library). So you see write(55, "[1,2,3,4]\n")

Let me try again. You do see write(55, "[1,2,3,4]\n", 10) and not { "syscall": "write", "params": { "fd": 55, "buf": "[1,2,3,4]\n", "len": 10 } } which would obviously be much better! It can be parsed by any modern language, and deserialization is cheap . Can we please rewrite strace with this in mind? Preferably in Rust.

How is the 2nd much better? Considering that a trace contains more than 1 single line?

Anyway they'd probably accept a patch for json output format. I don't think it's so difficult to do.

You can ask them in advance if they'd be willing to accept it, before starting to write it.

Re: Varlink – IPC to replace D-Bus gradually in systemd

#240

Post by dezgeg »

zzo38computer wrote: post #226

I think that there are problems with both Varlink and D-Bus. JSON is one of them but not only one. One of the problems with JSON is the data types it uses. There is no support for character codes other than Unicode (I think that "Unicode string types considered harmful"; they are not needed for handling text (even Unicode text)), and binary data must be encoded as hex or base64, which just makes it inefficient. There…

Yeah, using serialization format that doesn't even have obvious way of representing every possible Linux filename (nothing mandates filenames to be valid UTF-8, only thing they can't contain is NUL and '/') seems bad fit for low-level system services.

I can't wait to have apps and services crash and burn left and right due to accidentally (or maliciously) created non-UTF8 filenames in world readable directories.

Post reply on HN