Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spice: Fine-grained parallelism with sub-nanosecond overhead in Zig

github.com

Post reply on HN
Print view View original

31–40 of 56 posts

Re: Spice: Fine-grained parallelism with sub-nanosecond overhead in Zig

#31

Post by judofyr »

adrian_b wrote: post #30
x-complexity wrote: post #11

Earlier quoted context omitted.

> I can tell you “sub-nanosecond overhead” is misleading and marketing fluff If and only if (1-thread Spice - non-parallelized baseline) > 1ns, which their tests back up their claims. https://github.com/judofyr/spice/tree/main/bench

At your link it also says: "Spice shows subpar scalability: The speed-up of using 16 threads was merely ~11x" If that is true, then "Spice" is suitable only for small tasks, which can be completed at most in milliseconds, which can benefit from its low overhead, while for any bigger tasks something better must be used.

Author here.

I’d maybe phrase it as “Spice is not optimal” instead of “Spice is not suited for”, but yes, that’s the conclusion for this benchmark.

I’m hoping/assuming that for a more typical case (more CPU work being done) Spice will scale better, but I haven’t done the benchmark yet.

Re: Spice: Fine-grained parallelism with sub-nanosecond overhead in Zig

#32

Post by Veedrac »

chc4 wrote: post #29
Veedrac wrote: post #21

Earlier quoted context omitted.

I don't understand why it's unbelievable. They walk through the required logic to perform scheduling and dispatch; it looks incredibly simple and extremely suited to hiding on an OoO. The benchmark is summing down a binary tree which is going to give you a lot of space to hide what's at most a handful of instructions. There's obviously no lock contention because, like, look at the algorithm, what locks contending on…

> There's obviously no lock What? The threadpool has a shared mutex accessed by each worker thread, which is used for pushing work on heartbeat and for dequeuing work. https://github.com/judofyr/spice/blob/167deba6e4d319f96d9d67... "Adding more threads to the system will not make your program any slower" is an insane claim to be making for any lightweight task scheduling system, and trivially not true looking at this…

The README covers this. Its argument is persuasive. If your point is that the constant is badly tuned for theoretical 1000 core machines that don't exist, I'm not sure I care. A 100ns stall at most every 100us becoming more likely when you approach multiple hundreds of cores is hardly a disaster. In the context of the comment I replied to, the difference between 8 and 16 workers is literally zero, as the wakeups are spaced so the locks will never conflict.

Actually, if you did have a 32k core machine somehow with magical sufficiently-uniform memory for microthreading to be sensible for it, I think it's not even hard to extend the algorithm to work with that. Just put the workers on a 3D torus and only share orthogonally. It means you don't have perfect work sharing, but I'm also pretty sure it doesn't matter.

Re: Spice: Fine-grained parallelism with sub-nanosecond overhead in Zig

#33

Post by littlestymaar »

judofyr wrote: post #26
nirushiv wrote: post #4

I haven’t read through the code in detail but I can tell you “sub-nanosecond overhead” is misleading and marketing fluff. On first look, the measure seems to be some convoluted “time per thing” where the number of threads is far far smaller than the number of “thing”s

Author here. I knew that some people would react negatively to the term, but I can assure the intention is for you to have a better understanding of exactly how and when you should use Spice and Rayon. I would recommend reading the benchmark document: https://github.com/judofyr/spice/blob/main/bench/README.md . What people typically do when comparing parallel code is to only compare the sequential/baseline with a par…

Thanks for the answer, this part is particularly interesting indeed:

> Despite this being an utterly useless program we've learnt something that we can apply later on: Our smallest unit of work should probably be a bit bigger than ~7 ns before we reach for Rayon.

That's a very interesting project.

The big limitation I see with the current approach is that the usability of the library is much worth than what Rayon offers.

The true magic of Rayon is that you just replace `iter()` with `par_iter()` in you code and voilà! now you have a parallel execution. But yes it has some overhead, so maybe Rayon could try and implement this kind of scheduling as an alternative so that people pick what works best for their use-case.

Re: Spice: Fine-grained parallelism with sub-nanosecond overhead in Zig

#35

Post by mgaunard »

judofyr wrote: post #26
nirushiv wrote: post #4

I haven’t read through the code in detail but I can tell you “sub-nanosecond overhead” is misleading and marketing fluff. On first look, the measure seems to be some convoluted “time per thing” where the number of threads is far far smaller than the number of “thing”s

Author here. I knew that some people would react negatively to the term, but I can assure the intention is for you to have a better understanding of exactly how and when you should use Spice and Rayon. I would recommend reading the benchmark document: https://github.com/judofyr/spice/blob/main/bench/README.md . What people typically do when comparing parallel code is to only compare the sequential/baseline with a par…

You can just divide the speed-up by the number of cores, and that gives you the parallelization efficiency.

I've seen systems that can achieve 99% efficiency on thousands on nodes for real useful applications that involve non-trivial synchronization. Now that is an impressive feat.

Sure, there is probably some extra latency to get everything running, but for a sufficiently long program run, that is all irrelevant.

Re: Spice: Fine-grained parallelism with sub-nanosecond overhead in Zig

#36

Post by jnordwick »

imtringued wrote: post #27
jnordwick wrote: post #17

Earlier quoted context omitted.

Yesterday, he posted on Reddit and I expressed some concern with the benchmarks. The benchmarks are claiming 0.36 ns of overhead per call, but only the computing function. There is a second thread running doing the schedule that the overhead numbers don't include. It seems pretty clear he's running on a hyperthreaded 8 core machine (so 16 threads), I was guessing 3 Ghz, so that literally a single cycle of overhead. E…

The lock is acquired every 100 microseconds and it is expected that only one thread accesses it at a time.

I know. But I have no other explanation as to the latency increase that is seen as the number of threads increases. Do you have a better theory?

Re: Spice: Fine-grained parallelism with sub-nanosecond overhead in Zig

#38

Post by stareatgoats »

jd3 wrote: post #5
shoggouth wrote: post #3

List of limitations of the project: https://github.com/judofyr/spice?tab=readme-ov-file#limitati...

Want to state off the bat this this project is awesome and huge kudos to the author for spending their time, attention, and energy 1) working diligently to get this working at all and 2) sharing it with the broader HN community, who are generally known to by hyper-critical to a pedantic degree and/or overly pessimistic ( cough the initial Docker project Show HN thread cough ) I also really appreciate that the author…

> (cough the initial Docker project Show HN thread cough)

Docker was largely met with enthusiasm here when it was launched. I believe you must refer to how Dropbox was received — famously negatively, initially.

Re: Spice: Fine-grained parallelism with sub-nanosecond overhead in Zig

#39

Post by dagw »

jd3 wrote: post #5
shoggouth wrote: post #3

List of limitations of the project: https://github.com/judofyr/spice?tab=readme-ov-file#limitati...

Want to state off the bat this this project is awesome and huge kudos to the author for spending their time, attention, and energy 1) working diligently to get this working at all and 2) sharing it with the broader HN community, who are generally known to by hyper-critical to a pedantic degree and/or overly pessimistic ( cough the initial Docker project Show HN thread cough ) I also really appreciate that the author…

HN community, who are generally known to by hyper-critical to a pedantic degree and/or overly pessimistic (cough the initial Docker project Show HN thread cough)

So you made me dig up the announcement, and contrary to how you recall, it is almost universally positive.

https://news.ycombinator.com/item?id=5408002

Re: Spice: Fine-grained parallelism with sub-nanosecond overhead in Zig

#40

Post by WaffleIronMaker »

stareatgoats wrote: post #38
jd3 wrote: post #5

Earlier quoted context omitted.

Want to state off the bat this this project is awesome and huge kudos to the author for spending their time, attention, and energy 1) working diligently to get this working at all and 2) sharing it with the broader HN community, who are generally known to by hyper-critical to a pedantic degree and/or overly pessimistic ( cough the initial Docker project Show HN thread cough ) I also really appreciate that the author…

> (cough the initial Docker project Show HN thread cough) Docker was largely met with enthusiasm here when it was launched. I believe you must refer to how Dropbox was received — famously negatively, initially.

Yeah, that seems right:

April 5, 2007: "Show HN, Dropbox"

https://news.ycombinator.com/item?id=8863

Post reply on HN