Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

ekxide.io

Post reply on HN
Print view View original

21–30 of 53 posts

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#21

Post by elfenpiff »

sebastos wrote: post #13
elBoberido wrote: post #6

Earlier quoted context omitted.

Besides being written in Rust, the big difference is the decentralized approach. With iceoryx1 a central daemon is required but with iceoryx2 this in not the case anymore. Furthermore, more fine grained control over the resources like memory and endpoints like publisher. Overall the architecture is more modular and it should be easier to port iceoryx2 to even more platforms and customize it with 3rd party extension.…

I've been looking around for some kind of design documents that explain how you were able to ditch the central broker, but I haven't found much. Do you have breadcrumbs?

This is a longer story, but I'll try to provide the essence.

* All IPC resources are represented in the file system and have a global naming scheme. So if you would like to perform a service discovery, you take a look at the `/tmp/iceoryx2/services` list all service toml files that you are allowed to access and handle them.

* Connecting to a service means, under the hood, opening a specific shared memory identified via a naming scheme, adding yourself to the participant list, and receiving/sending data.

* Crashing/resource cleanup is done decentrally by every process that has the permissions to perform them.

* In a central/broker architecture you would have the central broker that checks this in a loop.

* In a decentralized architecture, we defined certain sync points where this is checked. These points are placed so that you check the misbehavior before it would affect you. For instance, when a sender shall send you a message every second but you do not receive it, you would actively check if it is still alive. Other sync points are, when an iceoryx2 node is created or you connect or disconnect to a service.

The main point is that the API is decentralized but you can always use it in a central daemon if you like - but you don't have to. It is optional.

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#22

Post by elfenpiff »

emmanueloga_ wrote: post #5

Looks great! From a quick glance it seems like it is a cross platform shared memory library. Maybe similar to this? [1]. Suggestion: would be cool to have a quick description of the system calls involved for each supported platform [2]. I'm guessing mmap on linux/osx and CreateFileMapping on Windows? -- 1: https://github.com/LiveAsynchronousVisualizedArchitecture/si... 2: https://github.com/eclipse-iceoryx/iceoryx2?t…

You guessed right. We have a layered architecture that abstracts this away for every platform. With this, we can support every OS as long as it has a way of sharing memory between processes (or tasks as some RTOSes are calling it) and you have a way of sending notifications.

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#23

Post by boolit2 »

I'm in robotics education and we mostly work with Python, to make life easier for students. I'd love to push for more Rust, but so far there's no point to it.

Multiprocess communication is something that we found lacking in Python (we want everything to be easily pip installable) and we ended up using shared memory primitives, which is a lot of code to maintain.

What is the main roadblock for iceoryx2 Python bindings? Is it something you are looking for contributors for?

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#24

Post by jcelerier »

nh2 wrote: post #17
abhirag wrote: post #16

Earlier quoted context omitted.

At $work we are evaluating different IPC strategies in Rust. My colleague expanded upon 3tilley's work, they have updated benchmarks with iceoryx2 included here[0]. I suppose the current release should perform even better. [0]: https://pranitha.rs/posts/rust-ipc-ping-pong/

Interesting that on Linux Unix Domain Sockets are not faster than TCP. People often say that the TCP stack overhead is high but this benchmark does not confirm that.

I'm curious about the benchmark. In my own for another network IPC library (https://GitHub.com/ossia/libossia) Unix sockets were consistently faster than the alternatives when sending the same payloads.

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#25

Post by hardwaresofton »

a_t48 wrote: post #18
hardwaresofton wrote: post #9

Been doing some IPC experiments recently following the 3tilley post[0], because there just isn't enough definitive information (even if it's a snapshot in time) out there. Shared memory is crazy fast, and I'm surprised that there aren't more things that take advantage of it. Super odd that gRPC doesn't do shared memory, and basically never plans to?[1]. All that said, the constructive criticism I can offer for this p…

In my experience shared memory is really hard to implement well and manage: 1. Unless you're using either fixed sized or specially allocated structures, you end up paying for serialization anyhow (zero copy is actually one copy). 2. There's no way to reference count the shared memory - if a reader crashes, it holds on to the memory it was reading. You can get around this with some form of watchdog process, or by othe…

Thanks for sharing here -- yeah these are definitely huge issues that make shared memory hard -- the when-things-go-wrong case is definitely quite hairy.

I wonder if it would work well as a sort of opt-in specialization? Start with TCP/UDS/STDIN/whatever, and then maybe graduate, and if anything goes wrong, report errors via the fallback?

I do agree it's rarely worth it (and same-machine UDS is probably good enough), but with the 10x gain essentially I'm quite surprised.

One thing I've also found that actually performed very well is ipc-channel[0]. I tried it because I wanted to see how something I might actually use would perform, and it was basically 1/10th the perf of shared memory.

[0]: https://crates.io/crates/ipc-channel

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#26

Post by fefe23 »

This smells like they are using shared memory, which is almost certainly a security nightmare. The way they are selling it makes me fear they aren't aware of what a time bomb they are sitting on.

Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith), or you do hardcore capturing (make a copy of each message in the framework before handing it off). Their web page mentions zero copy, so it's probably not the second one.

Also, benchmarks are misleading.

It's easy to get good latency if your throughput is so high that you can do polling or spin locks, like for example in benchmarks. But that's probably not a good assumption for general usage because it will be very inefficient and waste power and require more cooling as well.

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#27

Post by hardwaresofton »

abhirag wrote: post #16
hardwaresofton wrote: post #9

Been doing some IPC experiments recently following the 3tilley post[0], because there just isn't enough definitive information (even if it's a snapshot in time) out there. Shared memory is crazy fast, and I'm surprised that there aren't more things that take advantage of it. Super odd that gRPC doesn't do shared memory, and basically never plans to?[1]. All that said, the constructive criticism I can offer for this p…

At $work we are evaluating different IPC strategies in Rust. My colleague expanded upon 3tilley's work, they have updated benchmarks with iceoryx2 included here[0]. I suppose the current release should perform even better. [0]: https://pranitha.rs/posts/rust-ipc-ping-pong/

Excellent writeup! I performed just about the same test, but I didn't see 13M rps in my testing, shared memory went up to about 1M.

That said, I made sure to include serialization/deserialization (and JSON at that) to see what a realistic workload might be like.

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#28

Post by zbentley »

fefe23 wrote: post #26

This smells like they are using shared memory, which is almost certainly a security nightmare. The way they are selling it makes me fear they aren't aware of what a time bomb they are sitting on. Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith), or you do hardcore capturing (make a copy of each message in th…

> Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith)

There are all sorts of domains where mutually-trusted parties need IPC. Off the top of my head and in no particular order:

- Applications that pass validated data to/from captive subprocesses. Not everything is available as a natively-linked library. Not every language's natively-linked libraries are as convenient to reliably install as external binaries.

- Parallelism/server systems farming work out to forked (but not exec'd) subprocesses. Not everything needs setuid. Somtimes you just want to parallelize number crunching without the headache of threads (or are on a platform like Python which limits threads' usefulness).

- Replatforming/language transitions in data-intensive applications. Running the new runtime/platform in the same address space as the legacy platform can bring some hairy complexity, which is sidestepped (especially given the temporary-ness of the transitional state) with careful use of shared memory.

And aren't systems like Postgres counterpoints to your claim? My memory isn't the greatest, but IIRC postgres's server-side connections are subprocesses which interact with the postmaster via shared memory, no?

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#29

Post by billywhizz »

nh2 wrote: post #17
abhirag wrote: post #16

Earlier quoted context omitted.

At $work we are evaluating different IPC strategies in Rust. My colleague expanded upon 3tilley's work, they have updated benchmarks with iceoryx2 included here[0]. I suppose the current release should perform even better. [0]: https://pranitha.rs/posts/rust-ipc-ping-pong/

Interesting that on Linux Unix Domain Sockets are not faster than TCP. People often say that the TCP stack overhead is high but this benchmark does not confirm that.

the linux results are for a vm running on macos. not sure how useful that is. i certainly wouldn't draw any wider conclusions from them without trying to reproduce yourself. pretty sure they will be very different on bare metal.
Post reply on HN