How We Designed for Performance and Scale • Hacker News Board

Re: How We Designed for Performance and Scale

#61

Post by zobzu » Thu, Jun 11, 2015, 8:21 AM UTC

lorenzhs wrote: post #23

0x0 wrote: post #16

Earlier quoted context omitted.

The irc client irssi has something like this in /upgrade where it spawns a new binary but passing along active socket connections and their associated irc room states, I believe

WeeChat definitely does this. I don't know whether irssi supports it as well. Too bad WeeChat can't keep TLS connections open on /upgrade though

yeah weechat is one of the rare C apps to actually preserve state on upgrade (passes fds, saves state, load in new process, kill old process)

its not that hard to do if the program is made for this, even in C, its pretty hard to add when it isnt tho since you need a compat. layer.

Sometimes the new program simply has import functions for the old memory layout.

Re: How We Designed for Performance and Scale

#62

Post by jkarneges » Fri, Jun 12, 2015, 9:05 PM UTC

acidx wrote: post #57

jkarneges wrote: post #50

Earlier quoted context omitted.

Given the various efforts like libtask, lthread, boost coroutines, etc, it seems like the low-level assembly trickery ought to be isolated and standardized. Maybe some new methods (similar to the setcontext family) should be proposed to the glibc project. Something not at risk of deprecation.

Not sure if something like this should be part of glibc. Just recently there was an ABI break due to the fact that jmp_buf is exposed in the headers to allow embedding the struct[1]. [1] https://lwn.net/Articles/605607/ (Also, I ended up mixing up ucontext.h with setjmp.h in my comment above; Lwan uses ucontext.h as a fallback. There are coroutine implementations that will use setjmp/longjmp, or at the very least reu…

Glibc seems like a good place to put it to me. If everyone is preferring hand-coding assembly primitives that workaround suboptimal standard methods (ucontext), then my first instinct would be to look into making those existing methods more optimal.

(Note: in case it wasn't clear, I'm not suggesting we put all of coroutines into glibc, just the stack dancing stuff)

Re: How We Designed for Performance and Scale

#63

Post by the_why_of_y » Sat, Jun 13, 2015, 5:11 PM UTC

simi_ wrote: post #13

I had a feeling I've read about nginx before: http://aosabook.org/en/nginx.html The whole book is worth a read, although I found some sections painfully boring (perhaps my limited attention span is to blame).

There's a 3rd volume "The Performance of Open Source Applications" now, and it has a chapter on another high performance HTTP server, Warp:

http://www.aosabook.org/en/posa/warp.html

Interesting what kind of performance one can get out of GHC nowadays. Article says the authors of Warp had to implement a new parallel IO manager for GHC to get there, but that was merged into GHC 7.8.