How a fix in Go 1.9 sped up our Gitaly service by 30x

about.gitlab.com

71–72 of 72 posts

Re: How a fix in Go 1.9 sped up our Gitaly service by 30x

#71

Post by paulddraper » Sun, Jan 28, 2018, 2:49 AM UTC

arghwhat wrote: post #70

paulddraper wrote: post #69

Earlier quoted context omitted.

(1) The benchmark measured the point of discussion. (2) Even not using asynchronity (which Go is heralded for), processes take $ time seq 1000 | while read; do sleep 0; done real 0m1.644s user 0m1.065s sys 0m0.672s

1. No, the point was that fork(2)+exec(3)/spawning processes is an expensive way to run code, not how long it takes for the parent to be able to do something else. 2. Your new benchmark is better. However, it is still a useless microbenchmark, as it is an unrealistic best-case scenario. Your spawn of sleep is happening within a fresh subshell started by the pipe you made. fork(2) depends on things like VMM size and o…

> Each Gitaly server instance was fork/exec'ing Git processes about 20 times per second

> What's really wrong here is that they're apparently spawning processes like crazy.

Sounds like it depends on the use-case, rather then blanket "two dozen processes per second is clearly absurd".

Re: How a fix in Go 1.9 sped up our Gitaly service by 30x

#72

Post by arghwhat » Mon, Jan 29, 2018, 9:29 AM UTC

paulddraper wrote: post #71

arghwhat wrote: post #70

Earlier quoted context omitted.

1. No, the point was that fork(2)+exec(3)/spawning processes is an expensive way to run code, not how long it takes for the parent to be able to do something else. 2. Your new benchmark is better. However, it is still a useless microbenchmark, as it is an unrealistic best-case scenario. Your spawn of sleep is happening within a fresh subshell started by the pipe you made. fork(2) depends on things like VMM size and o…

> Each Gitaly server instance was fork/exec'ing Git processes about 20 times per second > What's really wrong here is that they're apparently spawning processes like crazy. Sounds like it depends on the use-case, rather then blanket "two dozen processes per second is clearly absurd".

Definitely. While fork(2) is expensive, a price is useless without also knowing the budget, and how expensive it is depends on the environment.

However, the problem in the posted article was indeed that spawning Git processes 20 times a second in that specific Go application was too much, and the fix was that Go replaced fork(2) with posix_spawn(3).

Post reply on HN