Live data from Hacker News

Why We Use Go

bravenewgeek.com

41–49 of 49 posts

Re: Why We Use Go

#41
post #34

Earlier quoted context omitted.

That's very sad for Rust, because Go's goroutines are amazingly awesome.

I'm pretty certain green threading can be designed as a library. However, the choices made by that library need not affect the language as a whole anymore.

Not easily--it has to be able to hook in at the linker level in order to deal with thread local storage properly.

Re: Why We Use Go

#42
post #34

Earlier quoted context omitted.

That's very sad for Rust, because Go's goroutines are amazingly awesome.

I firmly believe that moving away from M:N threading was the right decision for Rust. Even if nothing else, the fact that Rust FFI performance is thousands of times faster than cgo alone justifies the decision. There are numerous other reasons (fairness, I/O performance, performance of synchronization primitives, implementation complexity) why just using the OS scheduling is superior for a systems language. The prima…

How does having small initial stack sizes give you fast thread spawning?

Re: Why We Use Go

#43

Earlier quoted context omitted.

I firmly believe that moving away from M:N threading was the right decision for Rust. Even if nothing else, the fact that Rust FFI performance is thousands of times faster than cgo alone justifies the decision. There are numerous other reasons (fairness, I/O performance, performance of synchronization primitives, implementation complexity) why just using the OS scheduling is superior for a systems language. The prima…

How does having small initial stack sizes give you fast thread spawning?

Because you can pull the stack off the malloc free list bin, or even bump allocate it in a nursery.

Re: Why We Use Go

#44

Earlier quoted context omitted.

How does having small initial stack sizes give you fast thread spawning?

Because you can pull the stack off the malloc free list bin, or even bump allocate it in a nursery.

But you can allocate large stacks efficiently too. Is this something specific to how GC works?

Edit: Is it about having stacks smaller than a page, maybe not aligned to 4K and not fighting for the same cache lines?

Re: Why We Use Go

#45

Earlier quoted context omitted.

Because you can pull the stack off the malloc free list bin, or even bump allocate it in a nursery.

But you can allocate large stacks efficiently too. Is this something specific to how GC works? Edit: Is it about having stacks smaller than a page, maybe not aligned to 4K and not fighting for the same cache lines?

In general mallocs allocate small blocks a lot faster than they allocate large blocks, because small blocks qualify for free list binning and the OS kernel doesn't need to serve you as many pages. (It wasn't obvious to me either that this was the case, but try it and see for yourself!) :)

Re: Why We Use Go

#46

Earlier quoted context omitted.

But you can allocate large stacks efficiently too. Is this something specific to how GC works? Edit: Is it about having stacks smaller than a page, maybe not aligned to 4K and not fighting for the same cache lines?

In general mallocs allocate small blocks a lot faster than they allocate large blocks, because small blocks qualify for free list binning and the OS kernel doesn't need to serve you as many pages. (It wasn't obvious to me either that this was the case, but try it and see for yourself!) :)

At RethinkDB we managed our own free lists for stacks for this reason. One funny problem with the first implementation was that a stack could be born on one posix thread and die on another -- and certain threads would accumulate all the stacks, while the others would keep allocating them fresh.

Re: Why We Use Go

#47

Earlier quoted context omitted.

In general mallocs allocate small blocks a lot faster than they allocate large blocks, because small blocks qualify for free list binning and the OS kernel doesn't need to serve you as many pages. (It wasn't obvious to me either that this was the case, but try it and see for yourself!) :)

At RethinkDB we managed our own free lists for stacks for this reason. One funny problem with the first implementation was that a stack could be born on one posix thread and die on another -- and certain threads would accumulate all the stacks, while the others would keep allocating them fresh.

Yeah, caching works OK when your total thread count is relatively constant, but most thread spawning benchmarks do things like spawn 100000 threads, synchronize, and then shut them all down, in which case the thread high-water-mark is constantly increasing and your stack cache always misses.

Re: Why We Use Go

#48

Earlier quoted context omitted.

In general mallocs allocate small blocks a lot faster than they allocate large blocks, because small blocks qualify for free list binning and the OS kernel doesn't need to serve you as many pages. (It wasn't obvious to me either that this was the case, but try it and see for yourself!) :)

At RethinkDB we managed our own free lists for stacks for this reason. One funny problem with the first implementation was that a stack could be born on one posix thread and die on another -- and certain threads would accumulate all the stacks, while the others would keep allocating them fresh.

[deleted]

Re: Why We Use Go

#49

Earlier quoted context omitted.

At RethinkDB we managed our own free lists for stacks for this reason. One funny problem with the first implementation was that a stack could be born on one posix thread and die on another -- and certain threads would accumulate all the stacks, while the others would keep allocating them fresh.

Yeah, caching works OK when your total thread count is relatively constant, but most thread spawning benchmarks do things like spawn 100000 threads, synchronize, and then shut them all down, in which case the thread high-water-mark is constantly increasing and your stack cache always misses.

Could you find out the names and addresses of people making these benchmarks? If so, that seems like it would be a relatively easy problem to fix.

(Edit: I mean of course so we could allocate memory at their addresses for better locality, this is HN after all. Other ideas would need to be pulled to a different arena.)

Post reply on HN