Live data from Hacker News

Pingora, the proxy that connects Cloudflare to the Internet

blog.cloudflare.com

71–80 of 127 posts

Re: Pingora, the proxy that connects Cloudflare to the Internet

#71

Earlier quoted context omitted.

Wow ... 100 Gbit/s. Where do you work? That’s some serious traffic.

A german company building an app for watching linear TV. Netflix is actually serving 400Gbit/s per node and already have 800Gbit/s ready. I think we can scale our setup up to 200 Gbit/s but we are too small. Total traffic is ~2 Tbit/s. Most challenging is the missing support of QUIC/http3 and KTLS in Golang. Also 100G NIC supply chain is difficult. We use NVIDIA Connect-X 6, but it's impossible to get a version with…

Es liegt mir auf der Zunge welche Firma das ist.

I think it starts with a Wa ... you don’t have to say. I kind of remember to have been stumbled on a Twitter engineering ipv6 tweet. Maybe I m wrong.

For me it’s impressive to get so much data through a computer. But I have one question, what does count as a node, is a node like 1 machine with dual sockets, a lot of ram and a lot of nics or is it like multiple machines combined that act as 1 node like a whole 19 inch rack.

Re: Pingora, the proxy that connects Cloudflare to the Internet

#72

What are HTTP status codes greater than 599 used for in practice? It'd be interesting to see another Cloudflare blog post that just goes into detail on the weird protocol behaviour they've had to work around over the years. I imagine they have more insight into this than pretty much any other organisation on the planet.

Presumably custom statuses for app-to-app traffic or someone's weird API, etc.

Re: Pingora, the proxy that connects Cloudflare to the Internet

#73

> When crashes do occur an engineer needs to spend time to diagnose how it happened and what caused it. Since Pingora's inception we’ve served a few hundred trillion requests and have yet to crash due to our service code. > In fact, Pingora crashes are so rare we usually find unrelated issues when we do encounter one. Recently we discovered a kernel bug soon after our service started crashing. We've also discovered h…

Which aspect(s) of Rust do you think are most responsible for this? (e.g. borrow checker, memory safety, culture that attracts devs who care about reliability, etc)

A few things:

- I think memory safety is a baseline. You'll note that memory safe languages already tend to be much more reliable than non-memory-safe languages in general.

- Then you have the error handling. A lot of unreliability in my code in other languages comes from unhandled exceptions that only occur rarely. Rust generally puts all possible error conditions in the type signature of the function. Meaning it's actually feasible to handle every failure case.

- Speaking of unhandled exceptions, a lot of those in typed languages tend to be caused by null. Rust does not have null. Instead it has Option, and it is impossible to access the contents of an option without doing the equivalent of a null check. So that entire class of errors is gone.

- Both Result (used for error handling) and Option (used instead of null) are what Rust calls enums, and what are more generally called Sum Types. I think these are a huge deal. They allow you to safely represent data that may be one thing or another with very strict type checking. These are broadly very useful in API design, and in my experience lead to much more robust code than the class hierarchies you need in OOP languages or unions which lack the safety checks. (Aside: sum types would be quite simple to add to other languages. I have no idea why they haven't been added yet).

- Speaking of classes, inheritance is not supported. So that's a bunch of confusing code that just isn't possible to write. This can add a bit of boilerplate to Rust code, but it makes it more straightforward and less bug prone.

- You mentioned the borrow checker. That definitely helps. It's yet another tool that allows you to write APIs that cannot be misused. A great example would be Rust's Mutex type. It can prove at compile time that code does not hold on to references to the protected data beyond the duration that the lock is held.

- Speaking of Mutex. Rust's Send and Sync traits provide very good thread safety. You almost don't need to worry about thread safety at all in Rust. Most concurrency bugs are prevented by the compiler (you can still do things like cause data races).

- Newtypes allow you to check invariants once and then have the fact that they remain satisfied enforced by the type system.

- All type casts are explicit.

- Lots of other little things

One final thing that I think is often overlooked. Rust is strict, and all of these checks apply not only to the code your write, but to all of your dependencies. That means that Rust libraries tend to be much more reliable than libraries from other ecosystems. That probably is partly because of a culture of reliability. But it's also because the language itself makes it hard to write sloppy code. And that the code you are building on is likely to be reliable makes it both less effort and more worthwhile to make your own code reliable (including for library authors), leading to virtuous circle of reliable code.

Re: Pingora, the proxy that connects Cloudflare to the Internet

#74

Earlier quoted context omitted.

Wow ... 100 Gbit/s. Where do you work? That’s some serious traffic.

A german company building an app for watching linear TV. Netflix is actually serving 400Gbit/s per node and already have 800Gbit/s ready. I think we can scale our setup up to 200 Gbit/s but we are too small. Total traffic is ~2 Tbit/s. Most challenging is the missing support of QUIC/http3 and KTLS in Golang. Also 100G NIC supply chain is difficult. We use NVIDIA Connect-X 6, but it's impossible to get a version with…

Interesting, do you do a lot of processing in Golang or basically you just use it as a wrapper around sendfile[1] ?

1. https://man7.org/linux/man-pages/man2/sendfile.2.html

Re: Pingora, the proxy that connects Cloudflare to the Internet

#76
post #75

They don't say much on why not Envoy. It would be interesting to hear if there were concerns with it.

To me it looked like they didn't have anything technical against Envoy, they just didn't want to be dependent upon someone else for what is a core part of their product anymore. Which is entirely reasonable and a good idea.

Re: Pingora, the proxy that connects Cloudflare to the Internet

#77

Does anyone know why nginx used separate processes for workers, instead of threads? This post makes it sound like threads are the way to go, but presumably nginx had a reason for using processes back in the day.

Wikipedia says Nginx started in 2004. If you look at the state of things for threads and other "lots of sockets to deal with" things in that timeframe, you can see multiprocess was probably a safer choice, especially if you intended on running well on a variety of Unix or Unix like OSes. This page has some of that state captured pretty well: http://www.kegel.com/c10k.html

Re: Pingora, the proxy that connects Cloudflare to the Internet

#78
Besides comparing this to Nginx plus Lua (OpenResty), has Cloudflare compared it to Haproxy plus Lua or any other similar proxies.

The main issue for me with Rust is that it takes significantly more resources (time, space, memory, CPU) to build projects from source. Building Haproxy is comparatively quick and easy.

The haproxy plus lua static binary (musl, no pcre) I use is already growing rather large. I will bet that Pingora binaries using shared libraries will be at least twice, maybe three times the size.

Re: Pingora, the proxy that connects Cloudflare to the Internet

#79
post #38

Earlier quoted context omitted.

I had a very similar experience. Much smaller scale, but the service was keeping internal state and clients were connecting with a WebSocket. It could handle up to a million clients on one server and it practically never crashed. While I was writing it I had only hobby-level experience with Rust and I was also mentoring a colleague, so he wrote a big chunk of code as a total Rust noob.

Is this using Async Rust?

Yes, if you want to handle that many clients, you'll need to use async. It's not too bad to learn.

Re: Pingora, the proxy that connects Cloudflare to the Internet

#80

Besides comparing this to Nginx plus Lua (OpenResty), has Cloudflare compared it to Haproxy plus Lua or any other similar proxies. The main issue for me with Rust is that it takes significantly more resources (time, space, memory, CPU) to build projects from source. Building Haproxy is comparatively quick and easy. The haproxy plus lua static binary (musl, no pcre) I use is already growing rather large. I will bet th…

I really love haproxy, but I have seen segfaults and other memory errors thay wouldn't have happened if it had been written in rust (not that that was an option when haproxy was originally written). If you are going to write a new http software from scratch, I definitely think rust is a good language to use.

I would have liked to have seen more explanation of why they decided to build their own rather than use haproxy though.

Post reply on HN