Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Principles for Fast Tokio Applications

dial9-rs.github.io

Post reply on HN
Print view View original

61–70 of 72 posts

Re: Principles for Fast Tokio Applications

#61

Post by pjmlp »

duped wrote: post #29
jeffbee wrote: post #22

Earlier quoted context omitted.

The difference is nobody in the C++ community believes that a dominant asynchronous executor library exists, and there is not a pervasive belief that it would be helpful.

The "C++ community", if it even exists, barely believes in sharing code let alone any library being "dominant." They'd have to agree on a build system first, after all. But honestly that's a mischaracterization of the situation in Rust. Tokio is popular for networked service backends. If that's the wheelhouse you're in then yea it might look "dominant."

Of course we do, it is done via OS package managers, commercial libraries and SDKs.

More recently, via vcpkg and conan.

Re: Principles for Fast Tokio Applications

#62

Post by pjmlp »

duped wrote: post #43
ablob wrote: post #35

Earlier quoted context omitted.

You don't need a build system to share code. You can share with header files and respective (shared) object files regardless of the build system you're using. Likewise you could just share the source. None of this needs a build system.

I was just being a bit sardonic because the C++ ecosystem is so fragmented that something like tokio couldn't really exist. It would be one of three executors in boost, abseil, or folly, and you would never see the kind of downstream ecosystem build on top of them because C++ shops are allergic to external dependencies.

Any organisation that cares about security should be allergic to external dependencies, that is why companies like Nexus and JFrog exist, with companies paying to keep internal repos infrastructure in shape.

One just doesn't install willy nilly from the Internet into the CI/CD pipeline.

Well, they do, and then spend a few late nights when there is a bunch of CVE to fix.

Re: Principles for Fast Tokio Applications

#63

Post by saghm »

SwtCyber wrote: post #55
CoolestBeans wrote: post #40

Earlier quoted context omitted.

Tasks and channels is the way. You can get something that feels like programming a real preemptive concurrency model like BEAM languages or golang but with minimal overhead.

Give a task ownership of some state, communicate through channels and suddenly a lot of locking just disappears from the design

I've long pined for an ergonomic way of defining actors in Rust. I feel like there must be some way to abstract things that doesn't leak a bunch of a details into the mental model around how to think about starting/stopping/communicating between actors, but every time I've tried to figure it out (or use a solution someone else made) it ends up being way more complicated than it feels like it needs to be, and not worth it over writing a bunch of manual tasks wrapping private structs that have a bunch of channels in them. I feel like I've tried everything I could think of in terms of API design to make this work in a way that doesn't require users either having to learn a bunch of bespoke rules for the implementation or spend a lot of extra effort on manual boilerplate, including some very wacky things (like a macro where you pass a name and a function and it defines you a new macro with that name for spawning the actor task), but nothing ends up like I'd want.

Re: Principles for Fast Tokio Applications

#64

Post by hansvm »

jeffbee wrote: post #4
Tsarp wrote: post #2

One great use of agentic coding is being able to add and very granular tracing instrumentation to help with these sort of optimizations.

Also a great way to make sure that your app spends most of its time in observability overhead. For example even the latency histogram that the OP mentions is wildly expensive.

I'm not sure how other people are using LLMs for instrumentation, but IMO the layer you want running in prod is very different from what you want running for a one-off test. E.g., I have some code floating around which burns a pinned core on increasing a counter, with a little wrapper code around grabbing real timestamps at the beginning and end of a session and converting between the two units of time. It's helpful when microbenchmarking a very small unit of code as it actually behaves in a larger program (not perfect -- obviously tweaks the icache and pipeline behavior at a minimum -- but no measurement has zero tradeoffs, and you're always choosing which set of tradeoffs you prefer). An LLM can quickly instrument the call path I care about while I study this or that intervention. The ability to bang out a large amount of throwaway code is delightful.

Re: Principles for Fast Tokio Applications

#65

Post by saghm »

lkirkwood wrote: post #50
LoganDark wrote: post #47

Earlier quoted context omitted.

Aren't semaphores a more fundamental primitive that is trickier to get right? Otherwise, the mutex guards in Rust are very ergonomic.

Semaphores are more general for sure. The common semaphore is just a counter, when you lock it you decrement the counter by one atomically. Therefore a mutex is just a semaphore with a limit of 1. I wouldn't say they're much trickier to get right, maybe just less frequently applicable in e.g. general web io tasks.

I suspect the reason that semaphores aren't as common in Rust specifically is that once you go above one reference to something, you stop being able to mutate it safely. Mutexes work because "one thing writing to this means nothing else can read it" doesn't go against the borrow-checking grain, but as soon as you want to allow two references to something, you can't write to it safely, so it only works when you want to delay processing of non-shared data (which as you mention is not particularly common) or read-only processing (at which point you probably just want an RwLock to get shared read-only concurrent access).

Re: Principles for Fast Tokio Applications

#66

Post by saghm »

eru wrote: post #52
CoolestBeans wrote: post #40

Earlier quoted context omitted.

Tasks and channels is the way. You can get something that feels like programming a real preemptive concurrency model like BEAM languages or golang but with minimal overhead.

When you send a message in Erlang, nothing the recipient does with the message impacts anything on the sender side. That's good! In principle, they could have used something like copy-on-write for this, but in practice they really just make a copy of the bytes. Alas in Go, when you mutate what you received on a channel, you mutate the object the sender might still be holding. That's pretty annoying. It gets worse, be…

I honestly stopped taking Go channels seriously when I found out that reading from a closed channel is indistinguishable from reading the zero value, but writing to a closed channel will panic. I don't want to have to use booleans and write `true` in order to ping another task and have it tell the difference between getting pinged or being hung up on, and I really don't want to have to architect things so that I need to have my tasks know when the other side of a channel is closed before writing to it (or catch panics instead of using regular error handling) because that feels like it defeats the whole purpose of not needing direct knowledge of the state of the other side. It genuinely seems like those design decisions basically came from a desire to use special operators on channels rather than just having `send` and `receive` methods (or worse, to intentionally avoid "tedious" error handling/optional checks), which is mind-boggling.

I hadn't even considered the implications of sending a reference over a channel, but I'll add that to the existing reasons I have to never want to touch Go again.

Re: Principles for Fast Tokio Applications

#67

Post by duped »

pjmlp wrote: post #62
duped wrote: post #43

Earlier quoted context omitted.

I was just being a bit sardonic because the C++ ecosystem is so fragmented that something like tokio couldn't really exist. It would be one of three executors in boost, abseil, or folly, and you would never see the kind of downstream ecosystem build on top of them because C++ shops are allergic to external dependencies.

Any organisation that cares about security should be allergic to external dependencies, that is why companies like Nexus and JFrog exist, with companies paying to keep internal repos infrastructure in shape. One just doesn't install willy nilly from the Internet into the CI/CD pipeline. Well, they do, and then spend a few late nights when there is a bunch of CVE to fix.

Using an artifactory instance as the origin is not functionally different from installing something 'willy nilly' from the internet. It addresses a narrow range of threats while predominantly being more reliable and faster than public repositories.

It doesn't fix the actual problems with C++, which is that it's significantly more difficult to get and use external dependencies because of the compilation and linkage model of C++ libraries.

If C++ were as easy to build and link as modern programming languages you'd see the same kinds of tools as cargo, and the same kinds of ecosystem evolution as rust, like tokio. But you don't, because C++ code sucks to build, package, distribute, update, and reuse.

(I'm aware/have used conan/meson/vcpkg/etc - doesn't change my opinion).

Re: Principles for Fast Tokio Applications

#68

Post by pjmlp »

duped wrote: post #67
pjmlp wrote: post #62

Earlier quoted context omitted.

Any organisation that cares about security should be allergic to external dependencies, that is why companies like Nexus and JFrog exist, with companies paying to keep internal repos infrastructure in shape. One just doesn't install willy nilly from the Internet into the CI/CD pipeline. Well, they do, and then spend a few late nights when there is a bunch of CVE to fix.

Using an artifactory instance as the origin is not functionally different from installing something 'willy nilly' from the internet. It addresses a narrow range of threats while predominantly being more reliable and faster than public repositories. It doesn't fix the actual problems with C++, which is that it's significantly more difficult to get and use external dependencies because of the compilation and linkage mo…

It surely does, because in most companies that care about security it isn't a mirror, rather the only third party packages that developers are allowed to use beyond the standard library.

Additionally, making new packages available for consumption requires approval from IT and possibly legal, before they become available for consumption.

What is hard is people educated in scripting languages not wanting to learn about toolchains.

The moment Rust depends on other programming languages, we get a build.rs spaghetti file, depending on the knowledge of those writing it, or people throwing away Cargo altogether, and replacing it with Bazel, buck2 and co.

Re: Principles for Fast Tokio Applications

#69

Post by eru »

saghm wrote: post #66
eru wrote: post #52

Earlier quoted context omitted.

When you send a message in Erlang, nothing the recipient does with the message impacts anything on the sender side. That's good! In principle, they could have used something like copy-on-write for this, but in practice they really just make a copy of the bytes. Alas in Go, when you mutate what you received on a channel, you mutate the object the sender might still be holding. That's pretty annoying. It gets worse, be…

I honestly stopped taking Go channels seriously when I found out that reading from a closed channel is indistinguishable from reading the zero value, but writing to a closed channel will panic. I don't want to have to use booleans and write `true` in order to ping another task and have it tell the difference between getting pinged or being hung up on, and I really don't want to have to architect things so that I need…

Historically, Go was also really weird in having a very arrogant design: the language designers allowed themselves a lot of generic operators and structures (like arrays, map, channel, the 'make' function etc), but they were considered taboo for the hoi polloi which had to make do with the equivalent of void* pointers and runtime casting (the empty interface shenanigans).

I say historically, because they got generics a while ago.

Re: Principles for Fast Tokio Applications

#70

Post by CoolestBeans »

saghm wrote: post #63
SwtCyber wrote: post #55

Earlier quoted context omitted.

Give a task ownership of some state, communicate through channels and suddenly a lot of locking just disappears from the design

I've long pined for an ergonomic way of defining actors in Rust. I feel like there must be some way to abstract things that doesn't leak a bunch of a details into the mental model around how to think about starting/stopping/communicating between actors, but every time I've tried to figure it out (or use a solution someone else made) it ends up being way more complicated than it feels like it needs to be, and not wort…

Yeah the Actor model just doesn't fit well with Rust without feeling like a DSL. Tokio works well because Rust's ownership model overlays with tokio nicely. The language is already doing the hard data safety stuff for tokio, tokio just adds a lot of conveniences.

Erlang and Go make their concurrency models work because it is embedded into the fabric of the language. Neither cares about zero cost abstractions or minimal runtime (and runtime transparency). Both languages accept that there is a runtime that the developer cannot fully control as part of the deal for their concurrency models.

For Rust to have this, you have to break assumptions Rust developers have about writing Rust code. You would effectively be writing a runtime in Rust and then code that uses this Actor library would essentially run on it. But it wouldn't feel right because it would look like Rust code but feel like something else. That sort of heavy framework stuff doesn't mesh super well with Rust even if the language is capable of it.

Post reply on HN