Live data from Hacker News

“DBus is seriously screwed up”

thread.gmane.org

51–60 of 209 posts

Re: “DBus is seriously screwed up”

#51
I can make two observations and hypotheses from the profiling results, which agree with Linus' conclusion of "bad user-level code":

- Memory allocation/deallocation are taking the most time.

- All the percentages for each function are very small.

The former is a characteristic of code which heavily abuses dynamic allocation. It's surprising to see how many programmers are not aware of the overhead it adds and would malloc()/free() frivolously when something simpler would suffice. This is also often accompanied by copious amounts of unnecessarily copying data around. I've worked with small embedded systems where every use of dynamic allocation would need to be justified thoroughly in code reviews; perhaps these developers would benefit from being put through the same process.

The latter is a phenomenon which arises from "excessive modularity": the functionality of the system has been split into so many little pieces that the time each function contributes to the overall total is tiny. Instead of seeing an obvious "80% of the time is being spent here" that could easily be targeted for optimisation, that 80% is scattered amongst several dozen functions each taking 1-2% each. The bottleneck isn't concentrated in one area --- the whole system is uniformly inefficient. It's extremely difficult to optimise a system like this because nothing in particular stands out as being optimisable. I've had to optimise some large Java applications that were like this, and the solution was basically to remove most of the code and rewrite it to get rid of many chains of indirection.

Re: “DBus is seriously screwed up”

#52
post #15
post #6

Earlier quoted context omitted.

I agree, the OP title is a bit misleading. I use dbus every day on my laptop and it works totally fine - I don't notice its existence. I'm all for improvements, but I'm quite patient and quite happy to wait this one out.

I made a silly desktop app some time ago and it used DBUS to get notifications from NetworkMonitor when the system went online/offline. Nothing too fancy, very few lines of code. When I was implementing that, I managed to get several segfaults from my Python code. All together seemed a little bit fragile to me :( That was 5 years ago, things are probably better now (to be fair, I don't know what was causing the crash…

Two years ago I had to use DBus to connect to a Bluetooth device from Java.

In the end, after a few minutes of good work the connection between the adapter and the peripheral would timeout (or someone would go out of range), the whole thing would stall and we'd never get another message from BlueZ, be able to connect to any device ever again. It was without doubt the worst development experience I ever had.

Worse, you couldn't reload the dbus library and start over because then Java would scream and crash. So we had to restart the JVM, and BlueZ, etc.

I'm sure I was doing something wrong, but if I can't get it to work properly in a matter of weeks then it's not just my fault.

Re: “DBus is seriously screwed up”

#53
post #36

It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. Heck, I have worked on an algorithmic trading platform that in the limit of 5us receives market data, dedups it (multiple multicast streams for redundancy), uncompresses it (fricking zli…

> It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that.

You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.

Re: “DBus is seriously screwed up”

#54
post #24
post #17

Earlier quoted context omitted.

That was my first reaction as well. But to be fair, the kdbus people always said the real speedup will be seen for large packets. However, the real issue is how slow overall DBus seems to be according to those numbers. As Linus says earlier in the thread: "No way should it take 4+ seconds to send a 1000b message to back and forth 20k times." This is indeed rather shocking.

I would expect any half decent HTTP server to be able to handle more messages per second than that. If that is going to be the future fabric underlying all application ecosystem on Linux I would be as wary of it as Linus is. I will put it another way: if it can't be more performant than a simple HTTP client/server it doesn't deserve to be included into mainline kernel especially when the costs are so high.

Well, he's not claiming that kdbus is that slow. He's claiming that kdbus is faster not because it's in the kernel, but because it's not shitty code like userspace dbus (completely different implementation).

Re: “DBus is seriously screwed up”

#55

Earlier quoted context omitted.

COM is indeed an interesting beast and very useful. Being able to integrate other programs into yours, relying on COM to do so is very helpful. I don't know of a way on Linux or OSX to embed a word processor to work on documents but never show the user (but this might be due to my lack of knowledge about those systems - I would welcome being enlightened). I once wrote something that processed a plethora of Word docum…

LibreOffice uses UNO, which is their own version of COM: http://en.wikipedia.org/wiki/Universal_Network_Objects

Thanks! I'll take a look!

Re: “DBus is seriously screwed up”

#56
post #28

Earlier quoted context omitted.

To me, DBUS seems like a solution looking for a problem: The majority of the time there is no real reason to have such a complex layer of abstraction for IPC, when the system already provides much simpler alternatives. Then you need to read and reread and reread Havoc Pennington's posts on why he wrote d-bus in the first place: https://news.ycombinator.com/item?id=8648995 https://news.ycombinator.com/item?id=8649459…

Is there a document that explains the why's and wherefores of dbus? Aside from those posts.

You can probably find it here:

https://wiki.freedesktop.org/www/Software/dbus/

Re: “DBus is seriously screwed up”

#57
post #36

It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. Heck, I have worked on an algorithmic trading platform that in the limit of 5us receives market data, dedups it (multiple multicast streams for redundancy), uncompresses it (fricking zli…

> It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.

Nothing obvious about it.

If you look at the Linus's trace, it's all heap and mutex operations. That's just sloppy internal design full of concurrency bottlenecks and lots of in-memory cloning. You certainly don't mean to imply that both are the only way to implement a "security policy driven message bus" software, do you?

Re: “DBus is seriously screwed up”

#58
post #31

Earlier quoted context omitted.

IIRC the use case is to stream media over DBus, which currently no one is doing because it is too slow.

I don't get the use case, why can't the programs use DBus to negotiate the streaming over some low-overhead transport? Why are people so insistent on ramming a square peg into a round hole?

Well actually thats what kdbus would do anyway, all the large transfers will happen over memfd buffers...

Re: “DBus is seriously screwed up”

#59
post #36

It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. Heck, I have worked on an algorithmic trading platform that in the limit of 5us receives market data, dedups it (multiple multicast streams for redundancy), uncompresses it (fricking zli…

> But still, come on, guys...

libnetsnmp used to make close to a million calloc() calls when responding to a query for a number of CPU sockets on the machine. That was kind of amazing in its sheer brazenness.

Re: “DBus is seriously screwed up”

#60
post #22
post #5

> Since then I'm convinced that people who are inventing RPC solve non-existing problems. Many RPC solutions ultimately failed, because they were slow and overdesigned/complex. (e.g. CORBA, Network OLE/DCOM, Java RMI, XML-RPC/SOAP and other XML-based protocols, etc.) Whereas e.g. REST (if you call it even RPC) is just very simple and enough for most purposes. It's more like the concept of Component Object Model faile…

You do realize that pretty much whole of Windows is, underneath, based on COM, including most of the .Net-exposed system API's? I'll agree that COM and DCOM are... hairy, if you're implementing COM objects 'by hand'. But there's a reason for all of it, and I have yet to see an easier and more flexible way to write cross-language components. I could write a COM object in C++, call it from VBScript in a very simple com…

It is still working very well indeed. I was recently trying to add photos to iCloud programmatically on Windows. Guess what, there is a COM interface for that.

https://gist.github.com/tobiasviehweger/7a302b7179efb99082d8

Post reply on HN