It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. Heck, I have worked on an algorithmic trading platform that in the limit of 5us receives market data, dedups it (multiple multicast streams for redundancy), uncompresses it (fricking zli…
> It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.
“DBus is seriously screwed up”
61–70 of 209 posts
Re: “DBus is seriously screwed up”
#62Earlier quoted context omitted.
IIRC the use case is to stream media over DBus, which currently no one is doing because it is too slow.
So: if DBus was suddenly fast enough to stream media over, what applications would be using this feature, for what sets of streaming endpoints?
Re: “DBus is seriously screwed up”
#63I can make two observations and hypotheses from the profiling results, which agree with Linus' conclusion of "bad user-level code": - Memory allocation/deallocation are taking the most time. - All the percentages for each function are very small. The former is a characteristic of code which heavily abuses dynamic allocation. It's surprising to see how many programmers are not aware of the overhead it adds and would m…
When you have inclusive time it usually becomes easy to find that function that's spending 80% of the time in either itself or in the things it calls.
Re: “DBus is seriously screwed up”
#64It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. Heck, I have worked on an algorithmic trading platform that in the limit of 5us receives market data, dedups it (multiple multicast streams for redundancy), uncompresses it (fricking zli…
> It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.
The real problem is he doesn't give any data for volume/bandwidth processed in his case.
I don't see why a "security policy driven message bus" would be more expensive than what he describes (unzipping, business rules encorcement, analyzing, etc).
Re: “DBus is seriously screwed up”
#65> Since then I'm convinced that people who are inventing RPC solve non-existing problems. Many RPC solutions ultimately failed, because they were slow and overdesigned/complex. (e.g. CORBA, Network OLE/DCOM, Java RMI, XML-RPC/SOAP and other XML-based protocols, etc.) Whereas e.g. REST (if you call it even RPC) is just very simple and enough for most purposes. It's more like the concept of Component Object Model faile…
It's more like the concept of Component Object Model failed. For example OLE (the base technology behind DCOM) doesn't fly beside the legacy Office usage. True, but the idea of the self-registering COM object eventually seemed to limp across the finish line. Anyways. "Two key kdbus developers" ( ahem ) have said that the entire kernel signal mechanism should be deprecated because it's "too brittle and complex". When…
Re: “DBus is seriously screwed up”
#66Earlier quoted context omitted.
So what's the use case for sending large packets of data over dbus where the performance gains warrant integrating the dbus server to the kernel? Are they planning to migrate the X server to dbus?
IIRC the use case is to stream media over DBus, which currently no one is doing because it is too slow.
I really still don't see a good-enough scenario for this.
Re: “DBus is seriously screwed up”
#67Earlier quoted context omitted.
alternatives, solutions, or closure The link you posted has what I think is the right one: The problem is that probably this code is not needed at all. To me, DBUS seems like a solution looking for a problem: The majority of the time there is no real reason to have such a complex layer of abstraction for IPC, when the system already provides much simpler alternatives.
> DBUS seems like a solution looking for a problem: The majority of the time there is no real reason to have such a complex layer of abstraction for IPC, when the system already provides much simpler alternatives. Indeed. I just use shm_open()+mmap(MAP_SHARED) and go about sharing my data between processes, and have had no problems whatsoever. Then throw in sem_open() so only one process is locking the data at a time…
Is that sarcasm, or you don't know what dbus is used for?
Re: “DBus is seriously screwed up”
#68Some form of notification publish/subscribe functionality is necessary in modern Linux. But dbus isn't very UNIX-y. It's also strangely opaque. With pipelines, you can see how the plumbing works. With dbus, programs can interact in nonobvious ways. Even worse is polkit, a poor replication of Windows Group Policy. The most unixy solution is buried at the bottom of this thread, from Plan 9: https://news.ycombinator.com…
Re: “DBus is seriously screwed up”
#69> Since then I'm convinced that people who are inventing RPC solve non-existing problems. Many RPC solutions ultimately failed, because they were slow and overdesigned/complex. (e.g. CORBA, Network OLE/DCOM, Java RMI, XML-RPC/SOAP and other XML-based protocols, etc.) Whereas e.g. REST (if you call it even RPC) is just very simple and enough for most purposes. It's more like the concept of Component Object Model faile…
Re: “DBus is seriously screwed up”
#70Earlier quoted context omitted.
COM is indeed an interesting beast and very useful. Being able to integrate other programs into yours, relying on COM to do so is very helpful. I don't know of a way on Linux or OSX to embed a word processor to work on documents but never show the user (but this might be due to my lack of knowledge about those systems - I would welcome being enlightened). I once wrote something that processed a plethora of Word docum…
LibreOffice uses UNO, which is their own version of COM: http://en.wikipedia.org/wiki/Universal_Network_Objects
Read the Criticism section:
"XPCOM adds a lot of code for marshalling objects between different usage contexts (e.g. different languages). This leads to code bloat in XPCOM based systems. This was one of the reasons why Apple forked KHTML to create the WebKit engine (which is now used in several web browsers in various forms, including Safari and Google Chrome) over the XPCOM-based Gecko rendering engine for their web browser.
The Gecko developers are currently trying to reduce superfluous uses of XPCOM in the Gecko layout engine. This process is commonly referred to as deCOMtamination within Mozilla."
But my original top comment was about RPC and the compound document format (OLE as example), not about COM.
Apple had a compound document format too (death since 1997): http://en.wikipedia.org/wiki/OpenDoc
"OpenDoc's flexibility came at a cost. OpenDoc components were invariably large and slow. For instance, opening a simple text editor part would often require 2 megabytes of RAM or more, whereas the same editor written as a standalone application could be as small as 32 KB. This initial overhead became less important as the number of documents open increased, since the basic cost was for shared libraries which implemented the system, but it was large compared to entry level machines of the day. Many developers felt that the extra overhead was too large, and since the operating system did not include OpenDoc capability, the memory footprint of their OpenDoc based applications appeared unacceptably large. In absolute terms, the one-time library overhead was approximately 1 megabyte of RAM, at the time half of a low-end desktop computer's entire RAM complement.
Another issue was that OpenDoc had little in common with most "real world" document formats, and so OpenDoc documents could really only be used by other OpenDoc machines. Although one would expect some effort to allow the system to export to other formats, this was often impractical because each component held its own data. For instance, it took significant effort for the system to be able to turn a text file with some pictures into a Microsoft Word document, both because the text editor had no idea what was in the embedded objects, and because the proprietary Microsoft format was undocumented and required reverse engineering.
It also appears that OpenDoc was a victim of an oversold concept, that of compound documents. Only a few specific examples are common, for instance most word processors and page layout programs include the ability to include graphics, and spreadsheets are expected to handle charts. [...]
But certainly the biggest problem with the project was that it was part of a very acrimonious competition between OpenDoc consortium members and Microsoft. The members of the OpenDoc alliance were all trying to obtain traction in a market rapidly being dominated by Microsoft Office. As the various partners all piled in their own pet technologies in hopes of making it an industry standard, OpenDoc grew increasingly unwieldy. At the same time, Microsoft used the synergy between the OS and applications divisions of the company to make it effectively mandatory that developers adopt the competing OLE technology. In order to obtain a Windows 95 compliance logo from Microsoft, one had to meet certain interoperability tests which were quite difficult to meet without adoption of OLE technology, even though the technology was largely only useful in integrating with Microsoft Office. OpenDoc was forced to create an interoperability layer in order to allow developers to even consider adoption, and this added a great technical burden to the project."
And there were others too: http://en.wikipedia.org/wiki/Compound_document