Migrating Dropbox from Nginx to Envoy

#1

Post by SaveTheRbtz » Thu, Jul 30, 2020, 5:07 PM UTC

Migrating Dropbox from Nginx to Envoy
dropbox.tech

Re: Migrating Dropbox from Nginx to Envoy

#2

Post by dmicher » Thu, Jul 30, 2020, 6:53 PM UTC

I know some people might find it a little controversial, but I’m super excited about our load balancing future and that we probably have the biggest Envoy deployment in the world now. When we moved most of Dropbox traffic to Envoy, we had to seamlessly migrate a system that already handles tens of millions of open connections, millions of requests per second, and terabits of bandwidth. This effectively made us into one of the biggest Envoy users.

Re: Migrating Dropbox from Nginx to Envoy

#3

Post by Havoc » Thu, Jul 30, 2020, 8:45 PM UTC

Sensing a bit of a trend here. Didn't another major player recently make the same switch?

Re: Migrating Dropbox from Nginx to Envoy

#4

Post by SaveTheRbtz » Thu, Jul 30, 2020, 9:13 PM UTC

Havoc wrote: post #3

Sensing a bit of a trend here. Didn't another major player recently make the same switch?

I think the best slice of who's migrating to Envoy can be observed via EnvoyCon talks[1][2]:

* Lyft (of course)

* Spotify

* Stripe

* Square

* eBay

* Yelp

* Pinterest

Plus the support from major cloud providers: Google, Microsoft, and Amazon.

[1] https://envoyconna18.sched.com/ [2] https://envoycon2019.sched.com/

Re: Migrating Dropbox from Nginx to Envoy

#5

Post by dmicher » Thu, Jul 30, 2020, 9:20 PM UTC

Havoc wrote: post #3

Sensing a bit of a trend here. Didn't another major player recently make the same switch?

It may actually become a trend. For well known reasons:

- Community

- Nginx served us well for almost a decade. But it didn’t adapt to current development best-practices

- Operationally Nginx was quite expensive to maintain

- C++

- Observability and monitoring

etc...

Re: Migrating Dropbox from Nginx to Envoy

#6

Post by rdli » Thu, Jul 30, 2020, 9:29 PM UTC

Really great post. I'm glad the post in particular mentioned community, because I think in the end this is the huge advantage Envoy has over NGINX. NGINX, could, in theory, resolve all technical issues raised in the post. But the fundamental tension between the open source and commercial versions cannot be resolved.

(Disclosure: We use Envoy as part of Ambassador, and so of course we're big fans!)

Re: Migrating Dropbox from Nginx to Envoy

#7

Post by iampims » Thu, Jul 30, 2020, 9:29 PM UTC

Havoc wrote: post #3

Sensing a bit of a trend here. Didn't another major player recently make the same switch?

Slack announced they were going to switch.

Re: Migrating Dropbox from Nginx to Envoy

#8

Post by freedomben » Thu, Jul 30, 2020, 9:43 PM UTC

dmicher wrote: post #5

Havoc wrote: post #3

Sensing a bit of a trend here. Didn't another major player recently make the same switch?

It may actually become a trend. For well known reasons: - Community - Nginx served us well for almost a decade. But it didn’t adapt to current development best-practices - Operationally Nginx was quite expensive to maintain - C++ - Observability and monitoring etc...

I'd add another reason: so many people only use nginx as a reverse proxy, and the proxy configuration feels duct-taped on sometimes. Envoy being written as a proxy first makes it a better interface IMHO.

Re: Migrating Dropbox from Nginx to Envoy

#9

Post by Thaxll » Thu, Jul 30, 2020, 9:52 PM UTC

Havoc wrote: post #3

Sensing a bit of a trend here. Didn't another major player recently make the same switch?

It's a drop in the water compared to Nginx usage.

Re: Migrating Dropbox from Nginx to Envoy

#10

Post by caiobegotti » Thu, Jul 30, 2020, 9:58 PM UTC

I'm positively surprised that Dropbox (at least from what I understood from the post) didn't require lots of changes or patches on top of the upstream codebase of Envoy to migrate their traffic!