Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C • Hacker News Board

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#51

Post by elBoberido » Mon, Sep 30, 2024, 5:11 PM UTC

sischoel wrote: post #42

hardwaresofton wrote: post #9

Been doing some IPC experiments recently following the 3tilley post[0], because there just isn't enough definitive information (even if it's a snapshot in time) out there. Shared memory is crazy fast, and I'm surprised that there aren't more things that take advantage of it. Super odd that gRPC doesn't do shared memory, and basically never plans to?[1]. All that said, the constructive criticism I can offer for this p…

I was looking a bit at the code for the shared memory implementation in https://github.com/3tilley/rust-experiments/tree/master/ipc and the dependency https://github.com/elast0ny/raw_sync-rs . My last systems programming class was already a few years ago and I am a bit rusty, so I got some questions: 1. Looking at the code in https://github.com/elast0ny/raw_sync-rs/blob/master/src/even... ) it looks like we are using…

I'm not sure I fully understand what you mean? Do you assume we implemented the same approach for shared memory communication like described in the blog post?

If that’s the case, I want to reassure you that we don’t use locks. Quite the contrary, we use lock-free[1] algorithm to implement the queues. We cannot use locks for the reason you mentioned and also for cases when an application dies while holding the lock. This would result in a deadlock and cannot be used in a safety critical environment. Btw, there are already cars out there which are using a predecessor of iceoryx to distribute camera data in an ECU.

For hard realtime systems we have a wait-free queue. This gives even more guarantees. Lock-free algorithms often have a CAS loop (compare and swap), which in theory can lead to starvation but it's practically unlikely as long as your system does not run at 100% CPU utilization all the time. As a young company, we cannot open source everything immediately, so the wait-free queue will be part of a commercial support package, together with more sophisticated tooling, like teased in the blog post.

Regarding memory guarantees. There are essentially the same guarantees like what you have when sharing an Arc via a Rust channel. After publishing the producer releases the ownership to the subscriber and they have read-only access for as long as they hold the sample. When the sample is dropped by all subscriber, it will be released back to the shared memory allocator.

Btw, we also have an event signalling mechanism to not poll the queue but wait until the producer signals that new data is available. But this requires a context switch and it is up to the user to decide if it is desired to have.

[1]: https://www.baeldung.com/lock-free-programming

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#52

Post by anarazel » Mon, Sep 30, 2024, 6:20 PM UTC

zbentley wrote: post #48

fefe23 wrote: post #32

Earlier quoted context omitted.

If you use shared memory with a captive process, that process can probably hack you if it gets taken over by an attacker. I agree with your parallelism counter-argument in principle. However even there it would probably make sense to not trust each other, to limit the blast radius of successful attacks. In your next point the "careful" illustrates exactly my point. Using shared memory for IPC is like using C or C++ a…

> Using shared memory for IPC is like using C or C++ and saying "well I'll be careful then" Sort of. In the replatforming/codebase conversion case, the amount of care and labor needed to effectively use shared memory for communication is, in my experience, much less than the care and labor needed to make things work in the same address space (and the labor and cost-in-currency of making things work with sufficient pe…

> I'd be curious how much effort and specific shared-memory-IPC-expertise is required on the part of the Postgres maintainers to keep the shared memory layer capable and secure.

I don't think security plays a huge role for shared memory in postgres - if an attacker gains arbitrary code execution in one postgres backend, the installation is hosed. No need to go through SHM to escalate to other backends, there's easier ways.

WRT capable: There's definitely substantial costs due to using inter-process shared memory. But it's more an architectural cost, rather than something that everyone has to bear while just doing mostly unrelated hacking. Some features get harder, less flexible and require more code.

FWIW, there's some work towards moving towards a threaded connection model. Still some way to go, but I expect it to happen eventually.

Re: Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

#53

Post by CyberDildonics » Wed, Oct 02, 2024, 1:11 AM UTC

fefe23 wrote: post #26

This smells like they are using shared memory, which is almost certainly a security nightmare. The way they are selling it makes me fear they aren't aware of what a time bomb they are sitting on. Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith), or you do hardcore capturing (make a copy of each message in th…

Shared memory would be two processes that can already do whatever they want communicating with each other. What is it that you think is a 'security nightmare' ?

what a time bomb they are sitting on

You didn't give any real evidence of this or examples.

Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place?

Because you can have two or more different processes communicate asynchronously. They are in their own memory space and running on different threads. One doesn't crash the other. All they need to work together is data structures and data formats.

Don't forget that files are the original IPC.

Also, benchmarks are misleading.

Saying something is wrong is easy when you don't have anything to show that it's wrong.

that's probably not a good assumption for general usage

Then don't do it. Shared memory can use atomics, it can be totally lock free. You each process do checks that are just atomically reading and integer.