Live data from Hacker News

“DBus is seriously screwed up”

thread.gmane.org

61–70 of 209 posts

Re: “DBus is seriously screwed up”

#61
post #36

It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. Heck, I have worked on an algorithmic trading platform that in the limit of 5us receives market data, dedups it (multiple multicast streams for redundancy), uncompresses it (fricking zli…

> It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.

[deleted]

Re: “DBus is seriously screwed up”

#62
post #31

Earlier quoted context omitted.

IIRC the use case is to stream media over DBus, which currently no one is doing because it is too slow.

So: if DBus was suddenly fast enough to stream media over, what applications would be using this feature, for what sets of streaming endpoints?

KDBus was designed with sandboxed applications in mind. So use-case wise think android-style "use the camera app to take a picture/video", where the camera app transfers the media to another app over kdbus, thus without leaking information about the system.

Re: “DBus is seriously screwed up”

#63

I can make two observations and hypotheses from the profiling results, which agree with Linus' conclusion of "bad user-level code": - Memory allocation/deallocation are taking the most time. - All the percentages for each function are very small. The former is a characteristic of code which heavily abuses dynamic allocation. It's surprising to see how many programmers are not aware of the overhead it adds and would m…

The way you generally deal with the second problem is to have a profiler that can give you the inclusive time spent in a subroutine, not just the exclusive time. The inclusive time is the time spent in that sub and in subs it calls, the exclusive time is time spent only in that sub.

When you have inclusive time it usually becomes easy to find that function that's spending 80% of the time in either itself or in the things it calls.

Re: “DBus is seriously screwed up”

#64
post #36

It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. Heck, I have worked on an algorithmic trading platform that in the limit of 5us receives market data, dedups it (multiple multicast streams for redundancy), uncompresses it (fricking zli…

> It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.

>You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.

The real problem is he doesn't give any data for volume/bandwidth processed in his case.

I don't see why a "security policy driven message bus" would be more expensive than what he describes (unzipping, business rules encorcement, analyzing, etc).

Re: “DBus is seriously screwed up”

#65
post #5

> Since then I'm convinced that people who are inventing RPC solve non-existing problems. Many RPC solutions ultimately failed, because they were slow and overdesigned/complex. (e.g. CORBA, Network OLE/DCOM, Java RMI, XML-RPC/SOAP and other XML-based protocols, etc.) Whereas e.g. REST (if you call it even RPC) is just very simple and enough for most purposes. It's more like the concept of Component Object Model faile…

It's more like the concept of Component Object Model failed. For example OLE (the base technology behind DCOM) doesn't fly beside the legacy Office usage. True, but the idea of the self-registering COM object eventually seemed to limp across the finish line. Anyways. "Two key kdbus developers" ( ahem ) have said that the entire kernel signal mechanism should be deprecated because it's "too brittle and complex". When…

Except they're not wrong. We have a potentially infinite set of signals and data we'd like to communicate to programs and a very limited set of signals to do it with. I mean apache uses sigwinch as an exit code!

Re: “DBus is seriously screwed up”

#66
post #31
post #27

Earlier quoted context omitted.

So what's the use case for sending large packets of data over dbus where the performance gains warrant integrating the dbus server to the kernel? Are they planning to migrate the X server to dbus?

IIRC the use case is to stream media over DBus, which currently no one is doing because it is too slow.

I'd much prefer a an IPC that lets me pass a socket and use that socket on both ends, instead of this.

I really still don't see a good-enough scenario for this.

Re: “DBus is seriously screwed up”

#67
post #43

Earlier quoted context omitted.

alternatives, solutions, or closure The link you posted has what I think is the right one: The problem is that probably this code is not needed at all. To me, DBUS seems like a solution looking for a problem: The majority of the time there is no real reason to have such a complex layer of abstraction for IPC, when the system already provides much simpler alternatives.

> DBUS seems like a solution looking for a problem: The majority of the time there is no real reason to have such a complex layer of abstraction for IPC, when the system already provides much simpler alternatives. Indeed. I just use shm_open()+mmap(MAP_SHARED) and go about sharing my data between processes, and have had no problems whatsoever. Then throw in sem_open() so only one process is locking the data at a time…

>Indeed. I just use shm_open()+mmap(MAP_SHARED) and go about sharing my data between processes, and have had no problems whatsoever. Then throw in sem_open() so only one process is locking the data at a time. Wrap all that in a 2KB header and it's trivial to use in any project.

Is that sarcasm, or you don't know what dbus is used for?

Re: “DBus is seriously screwed up”

#68
post #32

Some form of notification publish/subscribe functionality is necessary in modern Linux. But dbus isn't very UNIX-y. It's also strangely opaque. With pipelines, you can see how the plumbing works. With dbus, programs can interact in nonobvious ways. Even worse is polkit, a poor replication of Windows Group Policy. The most unixy solution is buried at the bottom of this thread, from Plan 9: https://news.ycombinator.com…

Plan 9 in no way implies performance. Just because you can make it look like a filesystem, doesn't mean it actually is going to achieve anything special performance wise.

Re: “DBus is seriously screwed up”

#69
post #5

> Since then I'm convinced that people who are inventing RPC solve non-existing problems. Many RPC solutions ultimately failed, because they were slow and overdesigned/complex. (e.g. CORBA, Network OLE/DCOM, Java RMI, XML-RPC/SOAP and other XML-based protocols, etc.) Whereas e.g. REST (if you call it even RPC) is just very simple and enough for most purposes. It's more like the concept of Component Object Model faile…

Isn't Mozilla's XPCOM (the basis of Firefox and all Mozilla applications?) also based on COM?

https://en.wikipedia.org/wiki/XPCOM

Re: “DBus is seriously screwed up”

#70

Earlier quoted context omitted.

COM is indeed an interesting beast and very useful. Being able to integrate other programs into yours, relying on COM to do so is very helpful. I don't know of a way on Linux or OSX to embed a word processor to work on documents but never show the user (but this might be due to my lack of knowledge about those systems - I would welcome being enlightened). I once wrote something that processed a plethora of Word docum…

LibreOffice uses UNO, which is their own version of COM: http://en.wikipedia.org/wiki/Universal_Network_Objects

Mozilla has its own XPCOM too: http://en.wikipedia.org/wiki/XPCOM

Read the Criticism section:

"XPCOM adds a lot of code for marshalling objects between different usage contexts (e.g. different languages). This leads to code bloat in XPCOM based systems. This was one of the reasons why Apple forked KHTML to create the WebKit engine (which is now used in several web browsers in various forms, including Safari and Google Chrome) over the XPCOM-based Gecko rendering engine for their web browser.

The Gecko developers are currently trying to reduce superfluous uses of XPCOM in the Gecko layout engine. This process is commonly referred to as deCOMtamination within Mozilla."

But my original top comment was about RPC and the compound document format (OLE as example), not about COM.

Apple had a compound document format too (death since 1997): http://en.wikipedia.org/wiki/OpenDoc

"OpenDoc's flexibility came at a cost. OpenDoc components were invariably large and slow. For instance, opening a simple text editor part would often require 2 megabytes of RAM or more, whereas the same editor written as a standalone application could be as small as 32 KB. This initial overhead became less important as the number of documents open increased, since the basic cost was for shared libraries which implemented the system, but it was large compared to entry level machines of the day. Many developers felt that the extra overhead was too large, and since the operating system did not include OpenDoc capability, the memory footprint of their OpenDoc based applications appeared unacceptably large. In absolute terms, the one-time library overhead was approximately 1 megabyte of RAM, at the time half of a low-end desktop computer's entire RAM complement.

Another issue was that OpenDoc had little in common with most "real world" document formats, and so OpenDoc documents could really only be used by other OpenDoc machines. Although one would expect some effort to allow the system to export to other formats, this was often impractical because each component held its own data. For instance, it took significant effort for the system to be able to turn a text file with some pictures into a Microsoft Word document, both because the text editor had no idea what was in the embedded objects, and because the proprietary Microsoft format was undocumented and required reverse engineering.

It also appears that OpenDoc was a victim of an oversold concept, that of compound documents. Only a few specific examples are common, for instance most word processors and page layout programs include the ability to include graphics, and spreadsheets are expected to handle charts. [...]

But certainly the biggest problem with the project was that it was part of a very acrimonious competition between OpenDoc consortium members and Microsoft. The members of the OpenDoc alliance were all trying to obtain traction in a market rapidly being dominated by Microsoft Office. As the various partners all piled in their own pet technologies in hopes of making it an industry standard, OpenDoc grew increasingly unwieldy. At the same time, Microsoft used the synergy between the OS and applications divisions of the company to make it effectively mandatory that developers adopt the competing OLE technology. In order to obtain a Windows 95 compliance logo from Microsoft, one had to meet certain interoperability tests which were quite difficult to meet without adoption of OLE technology, even though the technology was largely only useful in integrating with Microsoft Office. OpenDoc was forced to create an interoperability layer in order to allow developers to even consider adoption, and this added a great technical burden to the project."

And there were others too: http://en.wikipedia.org/wiki/Compound_document

Post reply on HN