Painless NGINX Ingress

#1

Post by danielmartins » Wed, Sep 13, 2017, 2:37 PM UTC

Re: Painless NGINX Ingress

#2

Post by hltbra » Wed, Sep 13, 2017, 3:02 PM UTC

Cool read. I don't use Kubernetes but I learned a few things from this blog post that are applicable to my ECS environment.

The NGINX config part is tricky and it didn't come to mind that many programs will try to be smart about machine resources and it won't work in the container world as expected. This was a good reminder. OP didn't mention what Linux distro he's using and what are all of the OS-level configs he changed in the end of the day; I'd like to see that (was there any config not mentioned in the post?).

It's awesome that OP had lots of monitoring to guide him through the problem discovery and experimentation. I need more of this in my ECS setup. I didn't hop on the Prometheus train yet, by the way.

Re: Painless NGINX Ingress

#3

Post by rjcaricio » Wed, Sep 13, 2017, 3:08 PM UTC

Thanks for sharing your experience. I've got great insights to double check in my current environment.

Could you share which version of NGINX you found the issue with the reloads? Which version the fix was released?

PS.: I find it interesting/brave that you use a single cluster for several environments.

Re: Painless NGINX Ingress

#4

Post by danielmartins » Wed, Sep 13, 2017, 3:13 PM UTC

hltbra wrote: post #2

Cool read. I don't use Kubernetes but I learned a few things from this blog post that are applicable to my ECS environment. The NGINX config part is tricky and it didn't come to mind that many programs will try to be smart about machine resources and it won't work in the container world as expected. This was a good reminder. OP didn't mention what Linux distro he's using and what are all of the OS-level configs he ch…

> OP didn't mention what Linux distro he's using and what are all of the OS-level configs he changed in the end of the day.

I'm using Container Linux, and yes, I did a few modifications, but I intentionally left them out of the blog post as someone would be tempted to use them as-is.

I'll share more details in that regard if more people seem interested.

Re: Painless NGINX Ingress

#5

Post by danielmartins » Wed, Sep 13, 2017, 3:21 PM UTC

rjcaricio wrote: post #3

Thanks for sharing your experience. I've got great insights to double check in my current environment. Could you share which version of NGINX you found the issue with the reloads? Which version the fix was released? PS.: I find it interesting/brave that you use a single cluster for several environments.

> Could you share which version of NGINX you found the issue with the reloads? Which version the fix was released?

I'm using 0.9.0-beta.13. I first reported this issue in a NGINX ingress PR[1], so the last couple of releases are not suffering from the bug I reported in the blog post.

> I find it interesting/brave that you use a single cluster for several environments.

I'm not working for a big corporation, so dev/staging/prod "environments" are just three deployment pipelines to the same infrastructure.

As of now, things are running smoothly as they are, but I might as well use different clusters for each environment in the future.

[1] https://github.com/kubernetes/ingress/pull/1088

Re: Painless NGINX Ingress

#6

Post by odammit » Wed, Sep 13, 2017, 4:36 PM UTC

This is a great read. I know the single cluster for all env is something that is sort of popular but it's always made me uncomfortable for the reasons stated in the article but also for handling kube upgrades. I'd like to give upgrades a swing on a staging server ahead of time rather than go straight to prod or building out a cluster to test an upgrade on.

I tend to keep my staging and prod clusters identical, even names of services (no prod-web and stage-web, just web).

I'll set them up in different AWS accounts to clearly separate them and the only difference they have is the DNS name of the cluster and who can access them.

Edit: I suck at italicizing and grammar.

Re: Painless NGINX Ingress

#7

Post by fulafel » Wed, Sep 13, 2017, 5:19 PM UTC

Why does everyone use reverse proxies? It seems complex and inefficient. Why not serve xhr's and other dynamic content from the app server(s) and static content from a static webserver?

Re: Painless NGINX Ingress

#8

Post by philipcristiano » Wed, Sep 13, 2017, 5:24 PM UTC

fulafel wrote: post #7

Why does everyone use reverse proxies? It seems complex and inefficient. Why not serve xhr's and other dynamic content from the app server(s) and static content from a static webserver?

What would you use to provide a single endpoint to multiple instances of an app server?

Re: Painless NGINX Ingress

#9

Post by odammit » Wed, Sep 13, 2017, 5:26 PM UTC

fulafel wrote: post #7

Why does everyone use reverse proxies? It seems complex and inefficient. Why not serve xhr's and other dynamic content from the app server(s) and static content from a static webserver?

Off the top of my head: load balancing, hiding details of app servers, compressing responses and multivariate testing.

All of which could be done at the app server level sure, but then that would shift that complexity to your app and your developers.

Oh and job security, obviously.

Re: Painless NGINX Ingress

#10

Post by hardwaresofton » Wed, Sep 13, 2017, 5:35 PM UTC

Shameless plug! The insights in this article are pretty deep but if you're looking for just a clumsy step 1 to setting up the NGINX ingress controller on Kubernetes, check out what I wrote:

https://vadosware.io/post/serving-http-applications-on-kuber...

The most important thing that I found out while working on the NGINX controller was that you can just jump into it and do some debugging by poking around at the NGINX configuration that's inside it. There's no insight in there as deep as what's in this article, but for those that are maybe new to Kubernetes, hope it's helpful!