Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

su3.io

Post reply on HN
Print view View original

61–66 of 66 posts

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#61

Post by jeroenhd »

tredre3 wrote: post #52
zsoltkacsandi wrote: post #3

From a technical standpoint, these are always impressive projects, but I've always wondered: has anyone ever encountered a use case where the Caddy was the bottleneck?

In my experience Caddy has worse latency and throughput than nginx. I've set up a service that frequently sends 600MB/s (~5gbps) with nginx and the CPU is just chilling at 50%, but Caddy on that machine bottlenecks at 300MB/s despite using 100% of the CPU. AES hardware acceleration was enabled and functional on both software. This is high throughput that most people won't see, but it was also on a far beefier machine…

My experience mirrors that, but mostly because Caddy does HTTP/2 and http3 by default, natively. HTTP 1.1 is faster for bulk data transfers (while being worse at pipelining and latency in general), so I'm not surprised Caddy is taking more CPU in this case.

In my experience, in terms of latency, Caddy is a lot faster, every single time. I don't know what modifications I need to do to nginx to make it comparative but Caddy easily shaves half of the connection and transfer delays on my local network.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#62

Post by ksec »

tredre3 wrote: post #52
zsoltkacsandi wrote: post #3

From a technical standpoint, these are always impressive projects, but I've always wondered: has anyone ever encountered a use case where the Caddy was the bottleneck?

In my experience Caddy has worse latency and throughput than nginx. I've set up a service that frequently sends 600MB/s (~5gbps) with nginx and the CPU is just chilling at 50%, but Caddy on that machine bottlenecks at 300MB/s despite using 100% of the CPU. AES hardware acceleration was enabled and functional on both software. This is high throughput that most people won't see, but it was also on a far beefier machine…

I don't think Caddy being slower is a real problem. But Caddy, their developers and community refuse to believe Caddy is slow is a much bigger issue.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#63

Post by losfair »

kennethallen wrote: post #57

I read this post and your post introducing zeroserve. One of the main parts of the original pitch was "no separate config file and scripts". Now you're adding a config file separate from the scripts. Is the pitch now that it's like Caddy but runs eBPF scripts in userspace?

Hi - zeroserve Caddy compat works by compiling Caddyfile to C. Technically I can keep the compiler in a separate binary/repository - but it just feels unnecessary.

zeroserve only understand eBPF at runtime. It's always the source-of-truth.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#64

Post by tracker1 »

solid_fuel wrote: post #51
someothherguyy wrote: post #25

Earlier quoted context omitted.

> But hey, I didn’t code a Webserver so far - so what do I know If you limit the scope, its worth doing and might not take as much effort as you might think. You could possibly find some enjoyment and learn a few things doing so.

In college I had a networks class where the capstone project was writing a basic HTTP server in C. It's actually shockingly easy, especially if you're only supporting get, and fetch. Mine was something like 70 lines, and would just listen on 8080 and fork when it got a connection before checking for the requested file and sending it or a 404. I was immediately tempted to try adding something like CGI support but didn…

Seconded, HTTP and event HTTPS to some extent isn't so bad to wireup... HTTP2/3/QUIC is another story IMO though.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#65

Post by jauntywundrkind »

I was very taken by the other recent zeroserve headline that showed up, zeroserve: a zero-config web server you can script with eBPF:

> The twist is that you can drop eBPF programs into the tarball and they run on every request, in userspace, as sandboxed middleware - rewriting, authenticating, and rate-limiting requests, or reverse-proxying them to a backend when you want it to act as a gateway in front of your app.

https://su3.io/posts/introducing-zeroserve https://news.ycombinator.com/item?id=48425723 (272 points, 9d ago, 66 comments)

It's such opposite audiences being targeted here. eBPF is for super nerds, Caddy for "i just want a just works http server". The headline here doesn't do much for me: I don't care about Caddy compatibility and 3x doesn't mean much to me (Caddy feels like it targets friendly and easy not performance). But eBPF scripting? Ok that's cool, that's rad.

Re: Caddy compatibility for zeroserve: 3x throughput and 70% lower latency

#66

Post by bastawhiz »

10000truths wrote: post #44
bastawhiz wrote: post #38

The idea of jit compilation of a web server in a small project is pretty terrifying to me. The attack surface here is enormous. And for what? My back end on a single host isn't pumping at 35k qps. If each request is 500 bytes, 35k qps is nearly 20mbps sustained with zero other io (in each direction). And this is using only two threads! I think you'd be hard pressed to find an application where this is meaningfully us…

> The idea of jit compilation of a web server in a small project is pretty terrifying to me. The attack surface here is enormous. Does Spring Boot terrify you, then? Or Lua scripts in nginx? Or PHP? All of these use JIT compilation to run code that handles web requests. Attack surface is a property of the JIT implementation, not of JIT itself. And eBPF is specifically designed to be very simple to implement and audit…

There's a big difference between a JIT like luajit which has lots of attention and fuzzing and one that's used by essentially nobody.

Would you compile your daily driver kernel with someone else's hobby C compiler that essentially nobody else uses?

Post reply on HN