How a fix in Go 1.9 sped up our Gitaly service by 30x
about.gitlab.com
How a fix in Go 1.9 sped up our Gitaly service by 30x
1–10 of 72 posts
Re: How a fix in Go 1.9 sped up our Gitaly service by 30x
#2"Recompiling with Go 1.9 solved the problem, thanks to the switch to posix_spawn"
I never understood why so many people use fork() instead of POSIX spawn(). For example OpenJDK (Java) also does this as the default for starting a process. Which leads to interesting results when you use it on a OS which does do memory over committing like Solaris. Since the process briefly doubles in memory use with fork() your process will die with an out of memory error.
Re: How a fix in Go 1.9 sped up our Gitaly service by 30x
#3"Recompiling with Go 1.9 solved the problem, thanks to the switch to posix_spawn" I never understood why so many people use fork() instead of POSIX spawn(). For example OpenJDK (Java) also does this as the default for starting a process. Which leads to interesting results when you use it on a OS which does do memory over committing like Solaris. Since the process briefly doubles in memory use with fork() your process…
Typo: "does do" should read "doesn't do".
Re: How a fix in Go 1.9 sped up our Gitaly service by 30x
#4"Recompiling with Go 1.9 solved the problem, thanks to the switch to posix_spawn" I never understood why so many people use fork() instead of POSIX spawn(). For example OpenJDK (Java) also does this as the default for starting a process. Which leads to interesting results when you use it on a OS which does do memory over committing like Solaris. Since the process briefly doubles in memory use with fork() your process…
Because decades of written material about Unix says fork() is really cool (even though it isn't)?
Re: How a fix in Go 1.9 sped up our Gitaly service by 30x
#5It's often to remember the other point of view when you see huge performance differences.
> A bug in Go <1.9 was causing a 30x slowdown in our Gitaly service.
Re: How a fix in Go 1.9 sped up our Gitaly service by 30x
#6"Recompiling with Go 1.9 solved the problem, thanks to the switch to posix_spawn" I never understood why so many people use fork() instead of POSIX spawn(). For example OpenJDK (Java) also does this as the default for starting a process. Which leads to interesting results when you use it on a OS which does do memory over committing like Solaris. Since the process briefly doubles in memory use with fork() your process…
Because decades of written material about Unix says fork() is really cool (even though it isn't)?
fork was alright before other people tacked on multiples of cruft like threads and whatnot onto commercial unixes and they became mainstream. the current problem is that you don't want to have to copy all file descriptors if all you're going to do is call "exec" and reduce them to three: in, out, err.
for example, here's the caveats section from the macOS fork man page:
There are limits to what you can do in the child process. To be totally safe you should restrict your-
self to only executing async-signal safe operations until such time as one of the exec functions is
called. All APIs, including global data symbols, in any framework or library should be assumed to be
unsafe after a fork() unless explicitly documented to be safe or async-signal safe. If you need to use
these frameworks in the child process, you must exec. In this situation it is reasonable to exec your-
self.
Earlier in the game, copy-on-write had to be created for the same reasons.
Re: How a fix in Go 1.9 sped up our Gitaly service by 30x
#7"Recompiling with Go 1.9 solved the problem, thanks to the switch to posix_spawn" I never understood why so many people use fork() instead of POSIX spawn(). For example OpenJDK (Java) also does this as the default for starting a process. Which leads to interesting results when you use it on a OS which does do memory over committing like Solaris. Since the process briefly doubles in memory use with fork() your process…
because posix_spawn() in linux often calls fork(). I just looked at the manpage now and it says under some conditions it'd call vfork() instead, but I don't remember that being the case when I last looked at this (6-7 years ago?)
Re: How a fix in Go 1.9 sped up our Gitaly service by 30x
#8[deleted]
Re: How a fix in Go 1.9 sped up our Gitaly service by 30x
#9Each Gitaly server instance was fork/exec'ing Git processes about 20 times per second so we seemed to finally have a very promising lead.
What's really wrong here is that they're apparently spawning processes like crazy. Do they spawn a new process for each API call? That's like running CGI programs under Apache, like it's 1995.
Re: How a fix in Go 1.9 sped up our Gitaly service by 30x
#10"Recompiling with Go 1.9 solved the problem, thanks to the switch to posix_spawn" I never understood why so many people use fork() instead of POSIX spawn(). For example OpenJDK (Java) also does this as the default for starting a process. Which leads to interesting results when you use it on a OS which does do memory over committing like Solaris. Since the process briefly doubles in memory use with fork() your process…
because posix_spawn() in linux often calls fork(). I just looked at the manpage now and it says under some conditions it'd call vfork() instead, but I don't remember that being the case when I last looked at this (6-7 years ago?)
Nowadays, posix_spawn() calls clone() on linux, with the CLONE_VM flag, behaving much like vfork() as far as I can tell.
That means the child and parent process shares the memory (until exec() is performed).
Especially if the parent process is multi-threaded this avoids a whole lot of pagefaults that would occur if using fork() when another thread touches memory, possibly triggering a lot of copy-on-writes in the time window between calling fork() and the child calling exec()
Code: https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/uni...