Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

su3.io

Post reply on HN
Print view View original

51–60 of 66 posts

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#51

Post by solid_fuel »

someothherguyy wrote: post #25
augunrik wrote: post #17

Earlier quoted context omitted.

Because it passes more boundaries and stuff. But hey, I didn’t code a Webserver so far - so what do I know. :D AFAIK eBPF can be hardware offloaded. If you have the use case.

> But hey, I didn’t code a Webserver so far - so what do I know If you limit the scope, its worth doing and might not take as much effort as you might think. You could possibly find some enjoyment and learn a few things doing so.

In college I had a networks class where the capstone project was writing a basic HTTP server in C. It's actually shockingly easy, especially if you're only supporting get, and fetch.

Mine was something like 70 lines, and would just listen on 8080 and fork when it got a connection before checking for the requested file and sending it or a 404. I was immediately tempted to try adding something like CGI support but didn't have the time that semester.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#52

Post by tredre3 »

zsoltkacsandi wrote: post #3

From a technical standpoint, these are always impressive projects, but I've always wondered: has anyone ever encountered a use case where the Caddy was the bottleneck?

In my experience Caddy has worse latency and throughput than nginx. I've set up a service that frequently sends 600MB/s (~5gbps) with nginx and the CPU is just chilling at 50%, but Caddy on that machine bottlenecks at 300MB/s despite using 100% of the CPU. AES hardware acceleration was enabled and functional on both software. This is high throughput that most people won't see, but it was also on a far beefier machine than most people would use. Caddy would definitely be a bottleneck when serving media from a raspberry pi. My last attempt was in 2025, Caddy has probably improved since then.

That being said nginx has some terrible defaults so if you're just naively benchmarking it as a proxy out of the box, you might find Caddy to be better. For example nginx caches active request bodies (in and out) to temp files in many scenarios (to block the backend/upstream as little as possible), whereas Caddy is more of a transparent proxy.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#53

Post by TurdF3rguson »

denkmoon wrote: post #49
TurdF3rguson wrote: post #46

Earlier quoted context omitted.

It's a little suspicious. Why are they doing something that no other website in the world does? I was curious about zero-whatever but not enough to do whatever this is.

Plenty of sites do this, you just don't interact with them. Corp and govt intranets love this stuff.

Right, I've interacted with them when I had to for work. I wouldn't post any of them on HN though.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#55

Post by dilyevsky »

tredre3 wrote: post #52
zsoltkacsandi wrote: post #3

From a technical standpoint, these are always impressive projects, but I've always wondered: has anyone ever encountered a use case where the Caddy was the bottleneck?

In my experience Caddy has worse latency and throughput than nginx. I've set up a service that frequently sends 600MB/s (~5gbps) with nginx and the CPU is just chilling at 50%, but Caddy on that machine bottlenecks at 300MB/s despite using 100% of the CPU. AES hardware acceleration was enabled and functional on both software. This is high throughput that most people won't see, but it was also on a far beefier machine…

If it's http2 then Go's stdlib is pretty unoptimized to say the least. Huffman decoder is really cache unfriendly (pointer chasing) and I think allocation heavy too. Same probably goes for http1 and http3.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#56

Post by rciorba »

augunrik wrote: post #17
phillipseamore wrote: post #11

Earlier quoted context omitted.

Why? It's one of the most optimized HTTP servers ever. Anything that claims beating nginx in benchmarks should be treated with high suspicion. I think these zeroserve numbers are likely accurate but it doesn't have the features and module ecosystem of nginx so the margins aren't worth it for me.

Because it passes more boundaries and stuff. But hey, I didn’t code a Webserver so far - so what do I know. :D AFAIK eBPF can be hardware offloaded. If you have the use case.

This is not in-kernel eBPF, this is a userspace eNPF runtime, so I don't see why it would pass fewer boundaries

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#57

Post by kennethallen »

I read this post and your post introducing zeroserve. One of the main parts of the original pitch was "no separate config file and scripts". Now you're adding a config file separate from the scripts. Is the pitch now that it's like Caddy but runs eBPF scripts in userspace?

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#58

Post by athrowaway3z »

pbohun wrote: post #41

I looked into writing an http server based on iouring myself, but all the resources I could find said iouring is less safe from a cybersecurity perspective. Is there a safe way to use iouring for a webserver, or is libuv the better way to go, even though it has less performance?

A webserver shouldn't be calling io_uring internals without doing its own bound checking and http logic, nor should it be calling io_uring with remote-controlled crafted fd's or access pattern that might still have bugs in them.

At the level you'd be exposing io_uring (internals) via external http requests; it's security is perfectly fine.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#59

Post by graynk »

10000truths wrote: post #44
bastawhiz wrote: post #38

The idea of jit compilation of a web server in a small project is pretty terrifying to me. The attack surface here is enormous. And for what? My back end on a single host isn't pumping at 35k qps. If each request is 500 bytes, 35k qps is nearly 20mbps sustained with zero other io (in each direction). And this is using only two threads! I think you'd be hard pressed to find an application where this is meaningfully us…

> The idea of jit compilation of a web server in a small project is pretty terrifying to me. The attack surface here is enormous. Does Spring Boot terrify you, then? Or Lua scripts in nginx? Or PHP? All of these use JIT compilation to run code that handles web requests. Attack surface is a property of the JIT implementation, not of JIT itself. And eBPF is specifically designed to be very simple to implement and audit…

> Does Spring Boot terrify you, then?

It should.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#60

Post by jeroenhd »

iknowstuff wrote: post #31
cmgbhm wrote: post #21

Earlier quoted context omitted.

That’s likely just the side effect of supporting mtls. Mutual TLS came around at the same time as Microsoft did implicit network auth. Seemed magical at the time and so hare brained for eons of problems. The user side tls never caught on in most circles and still has the ancient sharp edges

Could probably buff it with passkeys these days https://www.passkeyprf.com/

mTLS supports some protocol level security guarantees that passkeys don't. Because the keys are exchanged during connection setup, there's no need for a login screen and Javascript middleware to begin the authentication process. mTLS is also easy to implement for APIs, you basically get authentication for free.

Unfortunately, browsers don't invest into making a good UI for mTLS. If browsers simply put their foot down and said "we will not permit websites to ask for a certificate if the request does not contain the proper requirements" like they do in passkeys, mTLS would be just as easy to use (and even easier to manage and rotate!).

When I ran mTLS auth on my intranet, I discovered that a lot of sites will use mTLS support to do fingerprinting, which means a lot of pages will open a blocking popup (sometimes multiple times) when I just want to read an article.

Post reply on HN