Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Make resilient Go servers using timeouts, deadlines and context cancellation

ieftimov.com

Post reply on HN
Print view View original

21–30 of 42 posts

Re: Make resilient Go servers using timeouts, deadlines and context cancellation

#21

Post by Twirrim »

regecks wrote: post #8

I tried running Go HTTP servers bare to the internet (after Cloudflare promoted doing so in a blog post), but went back to using a reverse proxy the next time. The main benefit seems to be convenience. I can upgrade and graceful-restart nginx instead of having to rebuild and redeploy the Go server (involving a full app restart). Not having to worry about goroutine leaks because some jerk decided to send the request l…

One cloud service I worked for had no end of quirks exposed by someone using a 28.8k modem to upload data (if it'd been 36.6k modem it'd have been fine). It wasn't causing impact on other customers (we already had stuff to handle "slow PUTters"), they were the only one getting a bunch of 500s, but it did expose a series of unrealised assumptions in the service components.

Re: Make resilient Go servers using timeouts, deadlines and context cancellation

#22

Post by awinter-py »

absolutely agree with the risk of slow clients saturating your connection limit

when doing DB work with these, I'm a little shakier -- once I start a multistep DB write, I probably want it to finish. Yes I can use a transaction to roll back the whole thing, but I think there are cases where rollback is wrong and I'd rather keep the write.

so while cancellation is cool, it's also a little fraught and hard to test.

Re: Make resilient Go servers using timeouts, deadlines and context cancellation

#23

Post by jlokier »

zzzcpan wrote: post #19
jrockway wrote: post #11

Earlier quoted context omitted.

You have to worry about the jerk sending requests at 1 byte per second no matter which webserver you use. It's always a problem to let an unlimited number of people ask for an unlimited amount of resources; it's just that things like goroutines are heavier than a file descriptor or a few bytes of RAM, so you'll notice wasted goroutines more quickly than wasted fds or memory. Typically, you need to consider the total…

> You have to worry about the jerk sending requests at 1 byte per second no matter which webserver you use. Not necessarily. It's just free webservers don't bother dealing with it, but there are plenty of simple approaches. Like just dropping connections that are sending requests slower than some threshold or dropping the slowest connection when some total number of connections is reached. Or more complicated, which…

> None of these are easy to implement with synchronous multithreaded networking code though, like in Go. Realistically it's only viable with asynchronous single threaded programming models or an actor model.

It's hard to see why synchronous multi-threaded code would find these things any more difficult than async or actor models.

All three models are equally able to access shared data structures to keep track of resource usage statistics, per-connection statistics, and timers.

OS kernels do this routinely, and are essentially multi-threaded on SMP architectures or with kernel pre-emption.

Re: Make resilient Go servers using timeouts, deadlines and context cancellation

#24

Post by zzzcpan »

jlokier wrote: post #23
zzzcpan wrote: post #19

Earlier quoted context omitted.

> You have to worry about the jerk sending requests at 1 byte per second no matter which webserver you use. Not necessarily. It's just free webservers don't bother dealing with it, but there are plenty of simple approaches. Like just dropping connections that are sending requests slower than some threshold or dropping the slowest connection when some total number of connections is reached. Or more complicated, which…

> None of these are easy to implement with synchronous multithreaded networking code though, like in Go. Realistically it's only viable with asynchronous single threaded programming models or an actor model. It's hard to see why synchronous multi-threaded code would find these things any more difficult than async or actor models. All three models are equally able to access shared data structures to keep track of reso…

Basically the reason is you can't just kill a thread that shares memory with other threads. Go doesn't even have an ability to kill goroutines, so your only choices is manual context tracking and manual cancellation in every piece of code. But if you are in a an event loop, for example, you can just destroy any client at any point. Same with actors, if you are in an actor, you can just kill other actors.

Re: Make resilient Go servers using timeouts, deadlines and context cancellation

#25

Post by JyB »

Almost all articles I've seen explaining the context pkg are done with net/http examples.

That's fine but I feel like it might not be the best introduction for a novice as a lot of concepts are mixed together and they might miss the bigger picture. Context is not just for web servers. You don't have to know how net/http works.

You could simply demonstrate the usefulness of the context package with showing how to properly clean up your program on a sigterm. Or even gracefully stop a long running operation so you are not afraid to stop/start your program at the "wrong" time.

Re: Make resilient Go servers using timeouts, deadlines and context cancellation

#26

Post by thwarted »

diamondo25 wrote: post #14
rakoo wrote: post #13

Earlier quoted context omitted.

The function you write after time.After should use the same context, and check its Done channel before continuing execution

So you have to propagate the Context, hm. IIRC, go test will panic the test case when it times out. Not exactly sure tho. It would be nice if there was a kind of 'abort' feature to clean up subroutines spun off this thread

The best there is with the context package is to make sure to call the cancel function given to you by contexts that have cancelation. Usually you do this via defer. The cancel function is a no-op if the context is finished otherwise. All this ends up doing though is making sure that things that clean themselves up know to clean themselves up eventually.

Re: Make resilient Go servers using timeouts, deadlines and context cancellation

#27

Post by cpuguy83 »

rakoo wrote: post #13
diamondo25 wrote: post #12

Ok, good, contexts are now making sure you can handle upcoming timeouts decided by an upper layer (caller function). But how about the time.After function? It'll still be running in the background? So you can still have a memory or 'processing power' leak?

The function you write after time.After should use the same context, and check its Done channel before continuing execution

Probably best not to use `time.After`, because it indeed starts a timer that you have no control over unless you are waiting for the full time.

Re: Make resilient Go servers using timeouts, deadlines and context cancellation

#28

Post by JyB »

diamondo25 wrote: post #14
rakoo wrote: post #13

Earlier quoted context omitted.

The function you write after time.After should use the same context, and check its Done channel before continuing execution

So you have to propagate the Context, hm. IIRC, go test will panic the test case when it times out. Not exactly sure tho. It would be nice if there was a kind of 'abort' feature to clean up subroutines spun off this thread

In most cases, at the inner-most level you end up calling some sort of external library (sql, api-client, ...) that will handle the Done() channel itself.

All you have to do is make sure is to pass to the library the context that carries your timeout or cancellation signal. The "rule" that everyone seems to follow is to always take as first argument a context.Context if your library handles cancellation.

Re: Make resilient Go servers using timeouts, deadlines and context cancellation

#29

Post by earthboundkid »

jrockway wrote: post #11
regecks wrote: post #8

I tried running Go HTTP servers bare to the internet (after Cloudflare promoted doing so in a blog post), but went back to using a reverse proxy the next time. The main benefit seems to be convenience. I can upgrade and graceful-restart nginx instead of having to rebuild and redeploy the Go server (involving a full app restart). Not having to worry about goroutine leaks because some jerk decided to send the request l…

You have to worry about the jerk sending requests at 1 byte per second no matter which webserver you use. It's always a problem to let an unlimited number of people ask for an unlimited amount of resources; it's just that things like goroutines are heavier than a file descriptor or a few bytes of RAM, so you'll notice wasted goroutines more quickly than wasted fds or memory. Typically, you need to consider the total…

You have to worry about slow requests somewhere in your stack, but with a good network architecture, you can assume the problem is solved at the open internet interface and ignore it inside your trusted zone.

Re: Make resilient Go servers using timeouts, deadlines and context cancellation

#30

Post by shhsshs »

awinter-py wrote: post #22

absolutely agree with the risk of slow clients saturating your connection limit when doing DB work with these, I'm a little shakier -- once I start a multistep DB write, I probably want it to finish. Yes I can use a transaction to roll back the whole thing, but I think there are cases where rollback is wrong and I'd rather keep the write. so while cancellation is cool, it's also a little fraught and hard to test.

In those rare cases you can choose not to propagate the same context through those operations. Only check for cancellation once the operations have all finished.
Post reply on HN