Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Local async executors and why they should be the default

maciej.codes

Post reply on HN
Print view View original

221–230 of 327 posts

Re: Local async executors and why they should be the default

#221

Post by lenkite »

tjoff wrote: post #182
morelisp wrote: post #179

Earlier quoted context omitted.

> But the second you don't think about memory you are doomed to write bad code anyway. Yeah but the failure modes are different. "Doomed" in C++ means now I've got worms all over my network. Pretty much a fully negative outcome. "Doomed" in Rust means I can't change it anymore until I relearn and refactor everything. If it's a pacemaker or rocket engine, that's probably ideal. But if it's not... "Doomed" in Go or Jav…

The promise with a good architecture is that the doom part is more infrequent though. And that when the Go/Java doom scenario bites you you'll have a whole lot of technical debt to pay back.

Dunno, tech debt is pretty easy to solve in Go/Java (easier in Java) as long as you dedicate some time to it. Excellent refactoring, code-coverage and instrumentation and profiling tools.

Re: Local async executors and why they should be the default

#222

Post by lossolo »

jacquesm wrote: post #211
lossolo wrote: post #206

Earlier quoted context omitted.

Sure it can, but all design patterns have their use cases. Generalizing and saying that one is just "terrible" (which your previous comment implied), and people "should learn" is just untrue and an ignorant take. Sometimes it's the only sane solution compared to others, considering project constraints.

Yes, that one is terrible compared to the alternatives. If there is one thing that really irritates me about the way we go about IT then it is that stuff that works and is reliable and well understood gets tossed and replaced by 'new shiny thing' which then eventually undergoes the same treatment. We are eternally stuck in reinventing wheels, rather than that we make real progress in for instance reliability and are…

> Yes, that one is terrible compared to the alternatives

No, that's exactly opposite of what I meant when I described what I'm working on, that all of the alternatives that you both with OP described are terrible in that specific case because of project constraints. I actually measured that!

As to the rest of your comment I fully agree, tired of all the hype cycles myself.

Re: Local async executors and why they should be the default

#223

Post by pdimitar »

nlitened wrote: post #193
pdimitar wrote: post #96

Just a personal take: after not coding with Rust for several months, I find it more and more difficult to return to an async code I was writing. The whole thing just reads... ugly and inconsistent. It needs too much already-accumulated knowledge. As the article correctly points out, you need a bunch of stuff that are seemingly unrelated (and syntactically speaking you would never guess they belong together). And as o…

Honestly, is 150k events per second even a meaningful number to talk about in performance context? I've got Clojure code ingesting 150k messages per second, processing, then outputting 30k msgs per second sitting at 47% CPU on a cheapo 70 $/mo VPS. Straightforward unoptimized C/C++ achieves millions of ZeroMQ messages processed per second in a single thread. Why use async or even Rust at all at such small loads?

I knew somebody will call me out since I didn't give details. :D

It was a 1-2 vCPU k8s pod which is still pretty impressive IMO. On my 20 CPU threads workstation I can easily achieve multi-million events per second. Even one of my Linux laptops can go at around 1 million but its ETH interface started overheating and couldn't sustain it longer.

> Why use async or even Rust at all at such small loads?

That's the better question, yeah. We had 200+ k8s pods and had to ingest a lot of data is the simple answer. Using anything except C++ or Rust would have made our cloud bill much bigger.

Though I have to admit that nowadays I would be very strongly tempted to try with Golang or even OCaml's new multithreaded runtime.

Re: Local async executors and why they should be the default

#224

Post by bombela »

packetlost wrote: post #178
surajrmal wrote: post #172

Earlier quoted context omitted.

Lower overhead. It's the same reason you would need to avoid a GC. It may not be for everyone, but that isn't a rust goal necessarily.

I'm aware that Rust's threading has higher overhead because they're system threads, but what about green/user threads? Is there intrinsic overhead to userspace threads that async doesn't have? I think every "task"/thread needs it's own callstack space, and you're paying scheduling overhead no matter what. Is there literature on the topic?

The short answer is that an async function in rust (and C++ and C#) is rewritten by the compiler to become a state machine (a struct + an enum to discriminate the current state + an union store local variables). You could do it by hand, though of course it's tedious and error prone. Hence why almost nobody writes software like that.

For the compiler, the state machine code is amenable to optimizations. You can find examples online where a chain of async function is optimized to a single call in the main.

In contrast green thread and the like are very similar to a kernel thread. The difference is they are lighter because there is less bagage to carry around. But composing green threads require an arguably light context switch.

Re: Local async executors and why they should be the default

#225

Post by packetlost »

lycopodiopsida wrote: post #214
packetlost wrote: post #178

Earlier quoted context omitted.

I'm aware that Rust's threading has higher overhead because they're system threads, but what about green/user threads? Is there intrinsic overhead to userspace threads that async doesn't have? I think every "task"/thread needs it's own callstack space, and you're paying scheduling overhead no matter what. Is there literature on the topic?

As far as I understand, green/user threads were on the table in rust team initially, but were sorted out because having green threads also means there is a runtime managing them which rust tries to avoid. What I do not understand is where the principal difference with async lies: it has no runtime, but you have to bring your own for it to work anyway in form of e.g. tokio. What rust goal conflicts with a similar solu…

I think it was, and Tokio appears to implement green threads as well with `tokio::spawn()`

Re: Local async executors and why they should be the default

#226

Post by pdimitar »

insanitybit wrote: post #106
pdimitar wrote: post #104

Earlier quoted context omitted.

I do that of course, and that's one of the easiest ways to use async Rust. In real projects you need much more however. F.ex. I had to code an example of how to add tasks to an already running pool of tasks and posted my findings here: https://github.com/dimitarvp/rust-async-examples/blob/main/e... (there's #2 as well with some more comments and a different approach). The fact that I needed to make a GitHub repo and…

I'd like to think that I've worked on some "real projects" and I've never run into these issues tbh. It's just hard for me to wrap my mind around. Like I said, I get wanting more libraries (although I think they exist? A local pool exists and you can spawn tasks in it, using tokio) but I'm just not seeing a fundamental problem. I found the code you wrote very confusing so I can see why, when you come back to it, you…

> but I'm just not seeing a fundamental problem.

Well, I mentioned it a few times in this thread: things are mixed in non-intuitive ways -- async/await, tokio, and various crates (including StreamExt which was not at all obvious and it took me a while to finally find mentioned in a few blog posts and GitHub gists). It just introduces friction and nowadays I am more averse to tech that requires more homework. Admittedly a personal preference of course but I don't think it's an invalid concern either.

> I found the code you wrote very confusing so I can see why, when you come back to it, you find it confusing.

Well, I needed such a pattern in a project that had to start as quickly as at all possible and only start and schedule the most critical work -- and then start adding all the other work as the first batch of work was already in progress.

Would you write it differently and if so, how?

Re: Local async executors and why they should be the default

#227

Post by morelisp »

marcosdumay wrote: post #218
morelisp wrote: post #188

Earlier quoted context omitted.

But you're arguing in circles. If attention is paid, if the architecture is good, all else like programmer salary/skill being equal, any of those languages (and plenty of others) will work fine. (And honestly, not a huge difference in the work to achieve that, a good design will look about the same in all four.) It's only an interesting question which to use if attention is not paid, or you don't want to invest enoug…

> any of those languages (and plenty of others) will work fine Well, C won't. Any large C code needs a decade or two of bug-fixing before it is safe, and that only happens if the developers are competent and wiling to fix it.

What do you think this comment added to the discussion?

Re: Local async executors and why they should be the default

#228

Post by packetlost »

bombela wrote: post #224
packetlost wrote: post #178

Earlier quoted context omitted.

I'm aware that Rust's threading has higher overhead because they're system threads, but what about green/user threads? Is there intrinsic overhead to userspace threads that async doesn't have? I think every "task"/thread needs it's own callstack space, and you're paying scheduling overhead no matter what. Is there literature on the topic?

The short answer is that an async function in rust (and C++ and C#) is rewritten by the compiler to become a state machine (a struct + an enum to discriminate the current state + an union store local variables). You could do it by hand, though of course it's tedious and error prone. Hence why almost nobody writes software like that. For the compiler, the state machine code is amenable to optimizations. You can find e…

Ah, that makes sense. Thank you for the info!

Re: Local async executors and why they should be the default

#229

Post by pdimitar »

insanitybit wrote: post #148
jacquesm wrote: post #140

Earlier quoted context omitted.

Sounds to me like you are dealing with this at the process boundary, more like actor and message based software. This is a mature approach and will serve you well. The typical context for stuff like async/await is code that should be running in a separate process so it can block but doesn't. Then it gets ugly fast.

Yes, I suspect you're right. I do not like building giant monoliths with internal task systems. I prefer using async/await on the inside and microservices with rpcs or, ideally, streams for communication. I think perhaps people are trying to push too much into a single process?

> I think perhaps people are trying to push too much into a single process?

I'll immediately spin that right back at you: people are trying to do too much via too many OS processes. ¯\_(ツ)_/¯

I like my single-OS-process apps that can distribute work internally (via the aforementioned Erlang "green threads"; they are not exactly that but close), or Rust's tokio, or Golang's goroutines and channels and WaitGroups.

I suppose we can all retreat to our corners, shrug and say "well, it's just how I prefer doing things" but I still find the many-OS-processes approach overrated and leaky. You can optimize the program pretty well but then you have to worry about whether the kernel will schedule and distribute work properly (i.e. on a 24-core server, will spawning 24 copies even with CPU core pinning work well?).

I personally prefer to dispense with these worries and just make self-contained apps / services and then be able to put them in any hosting, and deal with scaling issues either by further optimizations, or just bumping the server bill with $20.

Re: Local async executors and why they should be the default

#230

Post by dwaite »

cmrdporcupine wrote: post #139
kibwen wrote: post #136

Earlier quoted context omitted.

> A systems programming language should not have mandated a technique like this Rust doesn't mandate anything. It's unipinionated when it comes to concurrency; use whatever approach that you think is best. Async/await is just one option among many.

Its presence as a fundamental syntactical element is a fairly strong statement about its advisability.

Its presence is not a mandate. It is not a decorative syntactical element, but affects compiler code generation the same way as rust has _synchronous_ generators.

A developer is still free to spawn a thousand foot-guns, and I daresay the assumption would be that they made that choice knowing how to properly juggle them.

Post reply on HN