Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I turned Zig into my favorite language to write network programs in

lalinsky.com

Post reply on HN
Print view View original

131–140 of 150 posts

Re: How I turned Zig into my favorite language to write network programs in

#131

Post by loeg »

ori_b wrote: post #122
loeg wrote: post #39

Earlier quoted context omitted.

Even so. You're talking about storing and loading at least ~16 8-byte registers, including the instruction pointer which is essentially a jump. Even to L1 that takes some time; more than a simple function call (jump + pushed return address).

Which, with store forwarding, can be shockingly cheap. You may not actually be hitting L1, and if you are, you're probably not hitting it synchronously. https://easyperf.net/blog/2018/03/09/Store-forwarding and, section 15.10 of https://www.agner.org/optimize/microarchitecture.pdf

Are you talking about context switching every handful of cycles? This is going to be extremely inefficient even with store forwarding.

Re: How I turned Zig into my favorite language to write network programs in

#132

Post by pron »

hawk_ wrote: post #128
pron wrote: post #125

Earlier quoted context omitted.

It is true that in some languages there could be difficulties due to the language's idiosyncrasies of implementation, but it's not an intrinsic difficulty. We've implemented virtual threads in the JVM, and we've used the same Thread API with no issue.

Yep the JVM structured concurrency implementation is amazing. One thing I got wondering especially when reading this post on HN though is if stackless coroutines could (have) fit the JVM in some way to get even better performance for those who may care.

They wouldn't have had better performance, though. There is no significant performance penalty we're paying, although there's a nuance here that may be worth pointing out.

There are two different usecases for coroutines that may tempt implementors to address with a single implementation, but the usecases are sufficiently different to separate into two different implementations. One is the generator use case. What makes it special is that there are exactly two communicating parties, and both of their state may fit in the CPU cache. The other use case is general concurrency, primarily for IO. In that situation, a scheduler juggles a large number of user-mode threads, and because of that, there is likely a cache miss on every context switch, no matter how efficient it is. However, in the second case, almost all of the performance is due to Little's law rather than context switch time (see my explanation here: https://inside.java/2020/08/07/loom-performance/).

That means that a "stackful" implementation of user-mode threads can have no significant performance penalty for the second use case (which, BTW, I think has much more value than the first), even though a more performant implementation is possible for the first use case. In Java we decided to tackle the second use case with virtual threads, and so far we've not offered something for the first (for which the demand is significantly lower).

What happens in languages that choose to tackle both use cases with the same construct is that they gain negligible performance in the second use case (at best), but they're paying for that negligible benefit with a substantial degradation in user experience. That's just a bad tradeoff, but some languages (especially low-level ones) may have little choice, because their stackful solution does carry a significant performance cost compared to Java because of Java's very efficient heap memory management.

Re: How I turned Zig into my favorite language to write network programs in

#133

Post by aidenn0 »

vlovich123 wrote: post #34
aidenn0 wrote: post #30

I am still mystified as to why callback-based async seems to have become the standard. What this and e.g. libtask[1] do seems so much cleaner to me. The Rust folks adopted async with callbacks, and they were essentially starting from scratch so had no need to do it that way, and they are smarter than I (both individually and collectively) so I'm sure they have a reason; I just don't know what it is. 1: https://swtch.…

The research Microsoft engineers did on stackful vs stackless coroutines for the c++ standard I think swayed this as “the way” to implement it for something targeting a systems level - significantly less memory overhead (you only pay for what you use) and offload the implementation details of the executor (lots of different design choices that can be made).

> significantly less memory overhead

On an OS with overcommit, you might also only pay for what you use (at a page granularity), but this may be defeated if the stack gets cleared (or initialized to a canary value) by the runtime.

Re: How I turned Zig into my favorite language to write network programs in

#134

Post by aidenn0 »

boomlinde wrote: post #60
aidenn0 wrote: post #30

I am still mystified as to why callback-based async seems to have become the standard. What this and e.g. libtask[1] do seems so much cleaner to me. The Rust folks adopted async with callbacks, and they were essentially starting from scratch so had no need to do it that way, and they are smarter than I (both individually and collectively) so I'm sure they have a reason; I just don't know what it is. 1: https://swtch.…

One thing I would consider "unclean" about the zio approach (and e.g. libtask) is that you pass it an arbitrary expected stack size (or, as in the example, assume the default) and practically just kind of hope it's big enough not to blow up and small enough to be able to spawn as many tasks as you need. Meanwhile, how much stack actually ends up being needed by the function is a platform specific implementation detai…

Maybe I've been on x64 Linux too long, but I would just specify 8MB of stack for each fiber and let overcommit handle the rest. For small fibers that would be 4k per fiber of RSS so a million fibers is 4GB of RAM which seems fine to me?

Re: How I turned Zig into my favorite language to write network programs in

#135

Post by aidenn0 »

gpderetta wrote: post #80
zozbot234 wrote: post #50

Earlier quoted context omitted.

Yup, stackful fibers are an anti-pattern. Here's Gor Nishanov's review for the C++ ISO committee https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p13... linked from https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=10... . Notice how it sums things up: > DO NOT USE FIBERS!

And this is the rebuttal: https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2019/p08... There are downsides to stackful coroutines (peak stack usage for example), but I feed that p1364 was attacking a strawman: first of all it is comparing a solution with builtin compiler support against a pure library implementation, second it is not even comparing against the reference implementation of the competing proposal.

The TL;DR of that sums up my opinions pretty well.

As an aside, I know Rust would be unlikely to implement segmented stacks for fibers, given that they were burned by the performance implications thereof previously.

Re: How I turned Zig into my favorite language to write network programs in

#136

Post by aidenn0 »

RustSupremacist wrote: post #95
aidenn0 wrote: post #31

Earlier quoted context omitted.

Pre-1.0 Rust used to have infinitely growing stacks, but they abandoned it due to (among other things) performance reasons (IIRC the stacks were not collected with Rust's GC[1], but rather on return; the deepest function calls may happen in tight loops, and if you are allocating and freeing the stack in a tight loop, oops!) 1: Yes, pre-1.0 Rust had a garbage collector.

Rust still has garbage collection if you use Arc and Rc. Not a garbage collector but this type of garbage collection.

I'm going to veer into no-true-scottsman territory for a bit and claim that those don't count since they cannot collect cycles (if I'm wrong and they implement e.g. trial-deletion, let me know). This isn't just academic, since cyclic data-structures are an important place where the borrow-checker can't help you, so a GC would be useful.

Re: How I turned Zig into my favorite language to write network programs in

#137

Post by aidenn0 »

zozbot234 wrote: post #50
vlovich123 wrote: post #34

Earlier quoted context omitted.

The research Microsoft engineers did on stackful vs stackless coroutines for the c++ standard I think swayed this as “the way” to implement it for something targeting a systems level - significantly less memory overhead (you only pay for what you use) and offload the implementation details of the executor (lots of different design choices that can be made).

Yup, stackful fibers are an anti-pattern. Here's Gor Nishanov's review for the C++ ISO committee https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p13... linked from https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=10... . Notice how it sums things up: > DO NOT USE FIBERS!

Many of these issues go away if you control the compiler and runtime, which Rust does (and they needed to make changes to those to add async, so changes were inevitable).

Re: How I turned Zig into my favorite language to write network programs in

#138

Post by aidenn0 »

bccdee wrote: post #102
pjmlp wrote: post #54

Earlier quoted context omitted.

Also, it is about time to let go with GC-phobia. https://www.withsecure.com/en/solutions/innovative-security-... https://www.ptc.com/en/products/developer-tools/perc Note the > This video illustrates the use case of Perc within the Aegis Combat System, a digital command and control system capable of identifying and tracking incoming threats and providing the war fighter with a solution to address threats. Aegis, deve…

Real-time GCs can only guarantee a certain number of deallocations per second. Even with a very well-designed GC, there's no free lunch. A system which manages its memory explicitly will not need to risk overloading its GC.

I think you have that backwards; they can only guarantee a certain number of allocations per second (once the application hits steady-state the two are the same, but there are times when it matters)

Re: How I turned Zig into my favorite language to write network programs in

#139

Post by ori_b »

loeg wrote: post #131
ori_b wrote: post #122

Earlier quoted context omitted.

Which, with store forwarding, can be shockingly cheap. You may not actually be hitting L1, and if you are, you're probably not hitting it synchronously. https://easyperf.net/blog/2018/03/09/Store-forwarding and, section 15.10 of https://www.agner.org/optimize/microarchitecture.pdf

Are you talking about context switching every handful of cycles? This is going to be extremely inefficient even with store forwarding.

Sure, and so is calling a function every handful of cycles. That's a big part of why compilers inline.

Either you're context switching often enough that store forwarding helps, or you're not spending a lot of time context switching. Either way, I would expect that you aren't waiting on L1: you put the write into a queue and move on.

Re: How I turned Zig into my favorite language to write network programs in

#140

Post by grayhatter »

bccdee wrote: post #99
grayhatter wrote: post #17

Earlier quoted context omitted.

Oh, I guess that's a fair point. I didn't consider the change from `std.fs.openFile` to `std.Io.Dir.openFile` to be meaningful, but I guess that is problematic for some reason? You're of course correct here; but I thought it was reasonable to omit changes that I would describe as namespace changes. Now considering the audience I regret doing so. (it now does require nhe Io object as well, so namespace is inarticulate…

> I didn't consider the change from `std.fs.openFile` to `std.Io.Dir.openFile` to be meaningful, but I guess that is problematic for some reason? Because you explicitly said that existing code would continue to work without `std.Io`. > Code you write tomorrow, will likely have a new Io interface, because you want to use that standard abstraction. But, if you don't want to use it, all your existing code will still wor…

> Because you explicitly said that existing code would continue to work without `std.Io`.

Because I'm not conflating passing an Io object, (what everyone expects to be mandatory) and existing APIs moving into the Io namespace (an API change that can only be considered significant if you're trying to win an argument on reddit).

These are drastically different changes, and only one can be considered a meaningful change.

> I like Zig, but it does not have a stable API. That's just how it is.

The last 3 minor version upgrades, required a diff in all of my projects. All of them could have been fixed with exclusively sed -i to update namespaces. None of them required real attention, or logic changes.

In one repo I made the namespace changes in isolation. After some time I then went back and rewrote a few blocks to take advantage of the features, runtime speed improvements, and generally improved code quality granted from the new API.

I don't expect zig's API to be stable, and I regret it if my comment gave you a different impression. But I stand by my comments because I respectfully refuse to ignore the pragmatics of using zig. Calling moving a function between namespaces breaking API change can be argued as technically correct, but bordering on being intentionally misleading.

Post reply on HN