Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why We Love QUIC and HTTP/3

fastly.com

Post reply on HN
Print view View original

41–50 of 116 posts

Re: Why We Love QUIC and HTTP/3

#41

Post by the8472 »

zzzcpan wrote: post #36
windexh8er wrote: post #29

Earlier quoted context omitted.

> I actually disable QUIC myself, because I've noticed it slows everything down too much on some home routers. I'm curious how you measured and came to this conclusion since the design and most all metrics claim the opposite? And are you sure it isn't a bufferbloat issue rather than QUIC?

I was just browsing websites that were loading unusually slowly, so did the usual ping/mtr to investigate, which pointed to the router. From there and a bit of tcpdumping the cause turned out to be UDP traffic to Google from another person, who was watching videos I think.

your router shouldn't introduce noticable latency. If it does it either has a weak CPU or network queues that are too large. In the former case you need to upgrade hardware, in the latter you need a firmware that supports CAKE.

Re: Why We Love QUIC and HTTP/3

#43

Post by drewg123 »

simfoo wrote: post #38
drewg123 wrote: post #28

QUIC costs something like 2x to 4x as much CPU time to serve large files or streams per byte as compared to TCP. This is because the anti-middlebox protections also mean that modern network hardware and software offloads that greatly reduce CPU time cannot work with QUIC. When combined with the fact that QUIC is userspace, that's just deadly for performance. I'm talking about TSO, LRO (aka GRO), kTLS, and kTLS + hw e…

Wouldn't pretty much all of that overhead compared to TCP vanish if QUIC was implemented in the kernel?

No, due to the lack of TSO/LRO. Its my understanding that QUIC is designed to encrypt packet medata so that middle boxes cannot re-segment traffic. This same feature prevents NICs from doing TSO.

Re: Why We Love QUIC and HTTP/3

#44

Post by teacpde »

drewg123 wrote: post #28

QUIC costs something like 2x to 4x as much CPU time to serve large files or streams per byte as compared to TCP. This is because the anti-middlebox protections also mean that modern network hardware and software offloads that greatly reduce CPU time cannot work with QUIC. When combined with the fact that QUIC is userspace, that's just deadly for performance. I'm talking about TSO, LRO (aka GRO), kTLS, and kTLS + hw e…

Awesome analysis. This is first time that I read about the downsides of QUIC, curious that whether implementing it in userspace was a conscious trade off knowing the performance downside or Google/IETF wasn’t aware of the problem at all?

Re: Why We Love QUIC and HTTP/3

#45

Post by codys »

the8472 wrote: post #40
drewg123 wrote: post #28

QUIC costs something like 2x to 4x as much CPU time to serve large files or streams per byte as compared to TCP. This is because the anti-middlebox protections also mean that modern network hardware and software offloads that greatly reduce CPU time cannot work with QUIC. When combined with the fact that QUIC is userspace, that's just deadly for performance. I'm talking about TSO, LRO (aka GRO), kTLS, and kTLS + hw e…

> and sends it 1500b at a time sendmmsg (or the upcoming io_uring) let you send multiple UDP packets with a single syscall.

While this is useful, I don't think it would completely resolve the noted "tons of send syscalls" issue. QUIC performs it's own flow control and I don't think it can just send all the packers composing a file at once (all the time, at least)

Re: Why We Love QUIC and HTTP/3

#46

Post by simfoo »

drewg123 wrote: post #43
simfoo wrote: post #38

Earlier quoted context omitted.

Wouldn't pretty much all of that overhead compared to TCP vanish if QUIC was implemented in the kernel?

No, due to the lack of TSO/LRO. Its my understanding that QUIC is designed to encrypt packet medata so that middle boxes cannot re-segment traffic. This same feature prevents NICs from doing TSO.

Ok thanks, that makes sense. For anyone else wondering what TSO is, see https://en.wikipedia.org/wiki/Large_send_offload

But again, couldn't there be NICs with offloading QUIC capabilities? Maybe this could even be done with firmware updates (I don't know how much of the TCP offloading is done in real hardware)

Re: Why We Love QUIC and HTTP/3

#47

Post by seanwilson »

drewg123 wrote: post #28

QUIC costs something like 2x to 4x as much CPU time to serve large files or streams per byte as compared to TCP. This is because the anti-middlebox protections also mean that modern network hardware and software offloads that greatly reduce CPU time cannot work with QUIC. When combined with the fact that QUIC is userspace, that's just deadly for performance. I'm talking about TSO, LRO (aka GRO), kTLS, and kTLS + hw e…

> To serve the same amount of traffic with QUIC, you'd probably need multiple Xeon Gold CPUS. I guess that Google can afford this.

Can you explain more about how the negatives you mention weigh up against the positives? There isn't a net benefit somewhere? If not, can something be changed to give a better balance like a hybrid solution?

Re: Why We Love QUIC and HTTP/3

#48

Post by irq-1 »

drewg123 wrote: post #28

QUIC costs something like 2x to 4x as much CPU time to serve large files or streams per byte as compared to TCP. This is because the anti-middlebox protections also mean that modern network hardware and software offloads that greatly reduce CPU time cannot work with QUIC. When combined with the fact that QUIC is userspace, that's just deadly for performance. I'm talking about TSO, LRO (aka GRO), kTLS, and kTLS + hw e…

Two questions: can't large files continue to be served on HTTP2? and won't https://www.dpdk.org/ allow user-space network stacks to do segmentation, etc...? (Maybe it's too immature?)

Re: Why We Love QUIC and HTTP/3

#49

Post by mcguire »

"TCP Fast Open is a stellar example of one such modification to TCP: eight years after it was first proposed, it is still not widely deployed, largely due to middleboxes."

Anyone remember TTCP?

Re: Why We Love QUIC and HTTP/3

#50

Post by the8472 »

codys wrote: post #45
the8472 wrote: post #40

Earlier quoted context omitted.

> and sends it 1500b at a time sendmmsg (or the upcoming io_uring) let you send multiple UDP packets with a single syscall.

While this is useful, I don't think it would completely resolve the noted "tons of send syscalls" issue. QUIC performs it's own flow control and I don't think it can just send all the packers composing a file at once (all the time, at least)

If your server handles many connections simultanously you can still bundle a lot of packets in a single sendmmsg syscall, it can dispatch to a different destination address for each packet.
Post reply on HN