Live data from Hacker News

Bus1: a new Linux interprocess communication proposal

lwn.net

121–130 of 139 posts

Re: Bus1: a new Linux interprocess communication proposal

#121
post #53

Earlier quoted context omitted.

That's just a daemon. The reason this is in the kernel is that DBus is slow (according to Linus, because it could have been written better by monkeys with typewriters), and Lennart wanted it in the kernel/systemd, and Greg KH agreed, and Linus trusts Greg.

The announcement email from David Herrmann explains that they did consider a user-space implementation, and lists the reasons why they decided that it had to be in-kernel: https://lists.linuxfoundation.org/pipermail/ksummit-discuss/...

> o guarantee a global message order [...]

> o scale with the number of CPUs available [...]

It seems to me that these two goals are conflicting. If they give up the first goal then suddenly the need of a broker disappear.

edit: quoting from the message linked by parent.

edit2: by reading the description of the ordering guarantees, it seems that they actually only guarantee partial ordering.

Re: Bus1: a new Linux interprocess communication proposal

#122

Earlier quoted context omitted.

Did you read the ksummit email? There were eight reasons given, performance was one of them (the third one). Not the only one. Not the main one.

What would you say is the main driving rationale for bus1?

I would say it is to create the primitives needed for a multicast-capable (n-to-n) local IPC system. Obviously in such a way that it is safe, reliable, predictable, scalable, fast and lightweight.

Re: Bus1: a new Linux interprocess communication proposal

#123
post #53

Earlier quoted context omitted.

The announcement email from David Herrmann explains that they did consider a user-space implementation, and lists the reasons why they decided that it had to be in-kernel: https://lists.linuxfoundation.org/pipermail/ksummit-discuss/...

> o guarantee a global message order [...] > o scale with the number of CPUs available [...] It seems to me that these two goals are conflicting. If they give up the first goal then suddenly the need of a broker disappear. edit: quoting from the message linked by parent. edit2: by reading the description of the ordering guarantees, it seems that they actually only guarantee partial ordering.

One should think the goals are conflicting, but they are not. The secret is that the order is partial, as you say, but from userspace's point of view it is indistinguishable from a total order. The only messages that are not well ordered are messages that can never interact, so it is not possible to observe the lack of order from userspace. I wrote up how it works here: https://github.com/bus1/bus1/wiki/Message-ordering

Re: Bus1: a new Linux interprocess communication proposal

#124
post #8
post #5

Earlier quoted context omitted.

AFAIK, there's no good way to "broadcast" messages without bouncing through a user space co-ordinator. The current primitives just don't match this kind of need. Presumably, this inefficiency could be a bottleneck for some cases and eliminating the extra step might help, plus a kernel implementation could be simpler (no extra daemons, etc.)

I wrote a proof of concept that does multicast using the file system and inotify. A bit hacky and there are drawbacks, but it works.

That's clever, but there are reasons a purpose-built subsystem could be a better a fit:

* Inotify events are coalesced. Great for notifications, may be bad for discrete messages.

* The sender may want information about # of messages sent, synchronous delivery, etc.

Re: Bus1: a new Linux interprocess communication proposal

#125

Earlier quoted context omitted.

What would you say is the main driving rationale for bus1?

I would say it is to create the primitives needed for a multicast-capable (n-to-n) local IPC system. Obviously in such a way that it is safe, reliable, predictable, scalable, fast and lightweight.

You can create a multicast-capable (n-to-n) IPC system in such a way that is safe, reliable, predictable, scalable, fast and lightweight in user-space. Kernel is not required!

Re: Bus1: a new Linux interprocess communication proposal

#126

Earlier quoted context omitted.

I would say it is to create the primitives needed for a multicast-capable (n-to-n) local IPC system. Obviously in such a way that it is safe, reliable, predictable, scalable, fast and lightweight.

You can create a multicast-capable (n-to-n) IPC system in such a way that is safe, reliable, predictable, scalable, fast and lightweight in user-space. Kernel is not required!

How? We give a list of issues in the email.

Re: Bus1: a new Linux interprocess communication proposal

#127

Earlier quoted context omitted.

You can create a multicast-capable (n-to-n) IPC system in such a way that is safe, reliable, predictable, scalable, fast and lightweight in user-space. Kernel is not required!

How? We give a list of issues in the email.

* I don't see a time-slice accounting for the bus broker as an issue. If it's a system daemon, you can define away the problem. What's the use case for doing time-slice accounting where ignoring the system daemon's usage matters?

* for priority inheritance, have multiple daemons for each message priority level. Only privileges processes can use the high-priority message router, but can optionally use the low-priority message router when they want to inherit the priority of their sender.

* like I said, context switching is irrelevant since context switching is not the bottleneck in event latency nor in power usage.

* if FD accounting is important, move it out of the broker and make it p2p with broker coordination.

* kernel and LSM hooking feels like a feature looking for a use-case. If anyone really wants to do that, I'm sure the specific use case can be accommodated. Not every user space program need be trackable via LSM.

* for side channel and message ordering, why is this a core feature of the transport layer? I don't think all applications need this and if they do, something simpler and more specific to the application protocol could be developed. Generalized message ordering guarantees as part of the transport layer seems like it violates end-to-end design. A robust application protocol will end up ensuring this on its own anyway.

There you have it. I'm sorry but you don't have a strong case. Linus is even more opposed to this than I am and is probably better about system design than I am as well.

Re: Bus1: a new Linux interprocess communication proposal

#128

Earlier quoted context omitted.

Yep, I understand that not everybody can afford it. To me, it's was a pure, dirty capitalist investment thing: that system paid for itself hundreds of times over with all the time and effort I've saved not having to mess with it . Best money I've ever spent, what with all the time I've saved, and all the jobs I obtained with it. Have you looked at alternatives to GNU/Linux, like PC-BSD or FreeBSD? Did you find those…

I already explained: As much as I love Linux for some reasons, and would love to move away from it for many others (broaden my horizons, escape systemd), I play video games. Right now, games barely get a Linux release. If I'm on BSD or Solaris, you can forget it. And while LX-branded zones are good, I think BSD is a better system for my needs, and in any case I doubt they're good enough to get the games talking to my…

And I'd love to avoid putting another layer between the games and the hardware if I have to. I remember using Wine.

I agree, that makes perfect sense.

and in any case I doubt they're good enough to get the games talking to my GPU properly.

Now that's a whole different story: NVIDIA provides Solaris-native packages for their drivers, and they JustWork(SM). Way back in the day, NVIDIA engineers were stunned that they didn't have to produce a completely recompiled driver for every Solaris release: thanks to device driver interfaces/driver development kit[1], one could develop the driver on an ancient version of Solaris and have it work flawlessly on the latest-and-greatest one.

When I wrote that I'm using Mac OS X, that was the truth, but it wasn't the entire truth: I am typing this on my intel-based workstation I put together myself, with Solaris 10 running on it. And it has an NVIDIA GTX 980 accelerator, the top of the line model one could buy at the end of 2014. Downloaded the driver from NVIDIA's site, installed it with pkgadd(1M), and it JustWorks(SM)[2]:

  > pkginfo -l NVDAgraphics NVDAgraphicsr
   PKGINST:  NVDAgraphics
      NAME:  NVIDIA Graphics System Software
  CATEGORY:  system,graphics
      ARCH:  i386
   VERSION:  343.22,REV=2014.09.11.17.06
   BASEDIR:  /usr
    VENDOR:  NVIDIA Corporation
      DESC:  X and OpenGL Drivers for NVIDIA Quadro graphics
  INSTDATE:  Dec 04 2014 22:37
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:      188 installed pathnames
                  30 shared pathnames
                   2 linked files
                  42 directories
                   7 executables
              405482 blocks used (approx)

   PKGINST:  NVDAgraphicsr
      NAME:  NVIDIA Graphics System Device Driver
  CATEGORY:  system,graphics
      ARCH:  i386
   VERSION:  343.22,REV=2014.09.11.17.06
   BASEDIR:  /
    VENDOR:  NVIDIA Corporation
      DESC:  Kernel Drivers for NVIDIA Quadro graphics
  INSTDATE:  Dec 04 2014 22:37
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:        6 installed pathnames
                   3 shared pathnames
                   3 directories
               52151 blocks used (approx)
However, all of that is not to say you could play games on Solaris. If they were really optimized for Solaris, then they would fly, as Solaris is really, really, really fast, especially on intel. But since they aren't, and that's not realistic right now (and might never be realistic), I completely understand where you're coming from.

As for Mac, OS X has plenty of major and indie titles in the Apple app store. But, it's for people who are actually willing to pay for that.

[1] http://docs.oracle.com/cd/E19253-01/816-4854/

[2] http://www.geforce.com/drivers/results/105347

Re: Bus1: a new Linux interprocess communication proposal

#129

Earlier quoted context omitted.

How? We give a list of issues in the email.

* I don't see a time-slice accounting for the bus broker as an issue. If it's a system daemon, you can define away the problem. What's the use case for doing time-slice accounting where ignoring the system daemon's usage matters? * for priority inheritance, have multiple daemons for each message priority level. Only privileges processes can use the high-priority message router, but can optionally use the low-priority…

I don't get what you are trying to achieve with this discussion. You didn't provide any serious alternative, you just say that the issues don't matter to you.

Obviously, if you do not care about any of the features, then you should not be using bus1. But that does not mean that others don't care, so why are you objecting so strongly to us providing a solution to these problems?

Firstly, if you don't want a bus IPC with multicast messaging, then use something else. But if you DO want that, then you cannot ignore message ordering, that is the whole point (otherwise you could just use repeated unicast instead).

If you want message ordering you cannot split things into several daemons, or you break the order (unless you add some more communication on top). Same with p2p (side-channel) communication for FDs.

The broker is basically doing work on behalf of the sender, but without proper time-slice accounting it is not accounted accordingly. Obviously in many cases that does not matter, but you cannot simply ignore the problem if you want a general, lasting solution.

It is true that performance is not the main feature, but that is not to say that it does not matter at all (to anyone, ever), so you cannot simply brush it aside, proposing a solution that would have (almost by definition) hundreds of percent worse performance (as you are just doing more round-trips, even ignoring the context switching). To say that IPC latency is "not the bottle neck" only makes sense if you have a specific use-case in mind. With sufficiently low latency, we open up the possibility of using IPC for things that could only be done in-process today. This is not the MAIN purpose, but ignoring it is short-sighted.

I just got an email today from people asking about the LSM hooks, so there certainly is interest, and this (like the message order) is not something you can bolt on top.

I have not seen Linus comment on this, please give a link if you have. Whatever reservations he had about kdbus should not apply to bus1.

Look, there are plenty of things that could be improved on bus1, and I'm sure some careful review will find things we could and should do differently, but please at least spend more than five seconds looking into what we did and thinking about the problem. We want a multicast IPC system, but we want one done properly. You cannot simply say that doing one that is several times slower, doesn't do proper resource accounting, gets the message order wrong and cannot integrate with LSM is "good enough for me" and expect that to be the end of it.

Re: Bus1: a new Linux interprocess communication proposal

#130

Earlier quoted context omitted.

I already explained: As much as I love Linux for some reasons, and would love to move away from it for many others (broaden my horizons, escape systemd), I play video games. Right now, games barely get a Linux release. If I'm on BSD or Solaris, you can forget it. And while LX-branded zones are good, I think BSD is a better system for my needs, and in any case I doubt they're good enough to get the games talking to my…

And I'd love to avoid putting another layer between the games and the hardware if I have to. I remember using Wine. I agree, that makes perfect sense. and in any case I doubt they're good enough to get the games talking to my GPU properly. Now that's a whole different story: NVIDIA provides Solaris-native packages for their drivers, and they JustWork(SM). Way back in the day, NVIDIA engineers were stunned that they d…

I was really concerned with a) driver perf, and b) how the drivers worked in an LX-branded zone. Steam and the Humble Bundle will leave me set for games.

And now, because you didn't ask for it, Game Reccomendations:

Have you tried out TIS-100? You can get it through Steam, and maybe a few other places, it runs on Mac, and I'm willing to bet it'll run in an LX-brand. It bills itself as "the assembly language puzzler nobody asked for," and has a focus on minimal instruction count, maximum speed, and paralellism. You should know by this point if you're interested.

You also may want to check out Xonotic, which is an arena FPS with a relatively lively community, which may compile on Solaris. Think a mixture of Quake and UT.

Post reply on HN