Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Iggy.rs – building message streaming in Rust

blog.iggy.rs

Post reply on HN
Print view View original

21–30 of 63 posts

Re: Iggy.rs – building message streaming in Rust

#21

Post by vlovich123 »

steveklabnik wrote: post #17
vlovich123 wrote: post #14

I'm surprised about the selection of monoio. AFAIK it requires the usage of the nightly compiler which I've found to not be a great choice when maintaining a project.

It does require nightly, but it only uses five features, one of which could be removed by adding an external crate, and the others are like, not anything super wild, for the most part. I haven't looked too deeply into the code but they all make sense to me, for example one of them is a standard library API for creating uninitialized containers, which allows you to eliminate a copy. I have not compared monoio with glo…

It’s not that the features are unreasonable but more that anything that uses it as a dependency also inherits the nightly requirement which makes things a bit more unstable because nightly is a floating version.

I did compare with glommio a bit and disliked the file API in monoio but that’s probably more a matter of taste.

Re: Iggy.rs – building message streaming in Rust

#22

Post by spetz »

vlovich123 wrote: post #14

I'm surprised about the selection of monoio. AFAIK it requires the usage of the nightly compiler which I've found to not be a great choice when maintaining a project.

Monoio seems to be the most performant runtime, and actually easy to use - we have decided to go with "bleeding edge" approach, as it will still take at least a few months to implement io_uring and other optimizations, as we'll have to rewrite some of the core parts and most likely shift towards thread-per-core architecture.

Re: Iggy.rs – building message streaming in Rust

#23

Post by spetz »

mattmmatthews wrote: post #13

i'm a bit confused as to what exactly this is; a low-level, messaging system? what are the implications of it and the comparison of competitors? I'm obviously missing something. Could someone elaborate?

This is a message streaming infrastructure, similar to Kafka, Redpadna etc. You can think of it as a kind of a simple database (so-called Write Ahead Log), being responsible for appending the messages in a highly performant manner.

Re: Iggy.rs – building message streaming in Rust

#24

Post by spetz »

iddan wrote: post #3

Love the idea, and the blog post writing. The author really comes off humble, honest and a constructive project leader. Good luck!

Thank you, I'd like to mention that the team is really awesome, as they've decided to join me in my efforts, just to have some fun as well :)

Re: Iggy.rs – building message streaming in Rust

#25

Post by spetz »

jauntywundrkind wrote: post #15

It seems like such a sharp, smart move to start with QUIC. It provides really useful SCTP-like multi-streaming, which is a great base to start from. There's already so many good libraries the project or others can work from, and it'll just keep getting better (and better optimized, likely). https://github.com/xileteam/awesome-quic?tab=readme-ov-file#... This seems like such a natural place where of course having a so…

I've started with QUIC, as I wanted to try out something new. Still, TCP protocol we have in place is a bit faster than QUIC, but it might be due to the lack of some additional tuning or so. On a side note, QUIC on MacOS is slow, when compared to Linux.

Re: Iggy.rs – building message streaming in Rust

#26

Post by spetz »

emmanueloga_ wrote: post #7

Looks like a direct competitor to JetStream [1]? Impressive progress in less than a year of work! -- 1: https://docs.nats.io/nats-concepts/jetstream

JetStream, Kafka, Redpanda, RabbitMQ Streams, Fluvio - quite a few message streaming solutions out there. Thanks!

Re: Iggy.rs – building message streaming in Rust

#28

Post by spetz »

bsaul wrote: post #6

How fast can one realistically expect to be able to build decent quality system-level rust code, provided the person is already familiar with C-style programming languages and has 0 rust experience ?

I had 0 Rust experience and close to 0 of system programming experience (except playing with some lower level communication protocols etc. but nothing fancy), but I'd say that something like message streaming isn't the system programming - it's close to building the database, but nowhere close to making direct usage of low level system APIs.
Post reply on HN