Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Efficiency trades off against resiliency

blog.nelhage.com

Post reply on HN
Print view View original

21–30 of 66 posts

Re: Efficiency trades off against resiliency

#21

Post by threeseed »

HALtheWise wrote: post #18
kqr wrote: post #11

Earlier quoted context omitted.

> If a server is running at 100% CPU but 30% of that is spent performing background work, the amount of slack available for a sudden surge in demand is 30%, In this context, the first half of that sentence is usually interpreted as something like "every 10 days, the computer is capable of doing 10^8 tasks, and there arrives 7×10^7 time-critical tasks and 3×10^7 background tasks." As you can see, if the background tas…

I was assuming that it's option 2 (you can spin up new workers within 10 days). If you're using cloud compute this is almost always true. If you're building an on-prem cluster, I'm assuming you either spin up cloud workers for the extra load, have extra servers that are fully powered off and can be booted in a few minutes, or just physically order and install new hardware. Amazon can ship you a computer in 2 days, so…

The idea that you can run an on-premise cluster and just supplement it with cloud workers is a nice idea in theory. But the reality is that this never happens.

Because the two main reasons that a company would be running an on-premise cluster is (a) security and (b) cost. Both of which conflict with running a hybrid model because it is extremely complex and expensive to do so.

Re: Efficiency trades off against resiliency

#22

Post by anonymoushn »

pclmulqdq wrote: post #17

On the one hand, this is somewhat true, but on the other hand, there are lots of efficiencies you can get without trading resiliency, and the author sets up a few false dichotomies here. The author mentions JSON vs struct serialization as an example, but flatbuffers and protobufs (or any binary protocol format that builds in an ID and a version number) give you the same resiliency benefits mentioned here. You don't h…

This comes up a lot, do you have any benchmarks for protobuf deserialization that demonstrate that it is faster than json for the same structs? Protobuf seems much slower to decode due to the inability to know where an integer ends before one is done decoding it. Benchmarks I could find indicated protobuf decoding is maybe ~5x slower than json, but obviously this varies based on the payload and I'm not sure how much larger json encodings of the same data tend to be.

Re: Efficiency trades off against resiliency

#23

Post by akvadrako »

anonymoushn wrote: post #22
pclmulqdq wrote: post #17

On the one hand, this is somewhat true, but on the other hand, there are lots of efficiencies you can get without trading resiliency, and the author sets up a few false dichotomies here. The author mentions JSON vs struct serialization as an example, but flatbuffers and protobufs (or any binary protocol format that builds in an ID and a version number) give you the same resiliency benefits mentioned here. You don't h…

This comes up a lot, do you have any benchmarks for protobuf deserialization that demonstrate that it is faster than json for the same structs? Protobuf seems much slower to decode due to the inability to know where an integer ends before one is done decoding it. Benchmarks I could find indicated protobuf decoding is maybe ~5x slower than json, but obviously this varies based on the payload and I'm not sure how much…

How would you know where an integer in JSON ends without looking ahead? You won't even know if it is an integer or not. It seems like a more expensive operation than protobuf.

Re: Efficiency trades off against resiliency

#24

Post by anonymoushn »

akvadrako wrote: post #23
anonymoushn wrote: post #22

Earlier quoted context omitted.

This comes up a lot, do you have any benchmarks for protobuf deserialization that demonstrate that it is faster than json for the same structs? Protobuf seems much slower to decode due to the inability to know where an integer ends before one is done decoding it. Benchmarks I could find indicated protobuf decoding is maybe ~5x slower than json, but obviously this varies based on the payload and I'm not sure how much…

How would you know where an integer in JSON ends without looking ahead? You won't even know if it is an integer or not. It seems like a more expensive operation than protobuf.

You use about 2 SIMD instructions to compare 16 or 32 or 64 characters to whitespace, commas, and closing brackets, and one of these will be the character after the end of the integer literal. Theres no dependency between decoding the integer and tokenizing or decoding the next thing.

Re: Efficiency trades off against resiliency

#25

Post by ownagefool »

threeseed wrote: post #21
HALtheWise wrote: post #18

Earlier quoted context omitted.

I was assuming that it's option 2 (you can spin up new workers within 10 days). If you're using cloud compute this is almost always true. If you're building an on-prem cluster, I'm assuming you either spin up cloud workers for the extra load, have extra servers that are fully powered off and can be booted in a few minutes, or just physically order and install new hardware. Amazon can ship you a computer in 2 days, so…

The idea that you can run an on-premise cluster and just supplement it with cloud workers is a nice idea in theory. But the reality is that this never happens. Because the two main reasons that a company would be running an on-premise cluster is (a) security and (b) cost. Both of which conflict with running a hybrid model because it is extremely complex and expensive to do so.

Perhaps today this is true.

In 2007 I was working for a webhostet / nascent iaas provider, and we had workloads doing exactly that.

Specific example would be Celtic FC who had a baseline of dedicated, but would scale into our VMs during events, i.e. uefa cup games.

Re: Efficiency trades off against resiliency

#26

Post by sjducb »

pclmulqdq wrote: post #17

On the one hand, this is somewhat true, but on the other hand, there are lots of efficiencies you can get without trading resiliency, and the author sets up a few false dichotomies here. The author mentions JSON vs struct serialization as an example, but flatbuffers and protobufs (or any binary protocol format that builds in an ID and a version number) give you the same resiliency benefits mentioned here. You don't h…

I would question if the serialisation code speed actually matters.

RAM and CPU are much much faster than an SSD so almost all (98%) of the time will be spent writing to disk.

To speed it up you have to reduce the file size so that it can be written to disk faster. I think the fastest way would be a gzipped JSON file.

You can do all of the converting to JSON and compression while you're waiting for the hard disk to finish writing.

Re: Efficiency trades off against resiliency

#29

Post by regularfry »

Some of these examples are trivially equivalent to queues and queueing theory tells us that 100% utilisation results in unbounded queue lengths, and is therefore bad. The interesting thing would be to try to see if these other situations that on the surface sort of smell similar (serialisation and distribution systems, for instance) are actually isomorphic to the extent that the same proofs apply.

Re: Efficiency trades off against resiliency

#30

Post by twawaaay »

HALtheWise wrote: post #7

While I generally agree that the pattern this article describes is real (there's some degree of tradeoff between robustness and efficiency), I've frequently seen engineers fall back on that logic rather than actually thinking about the specific problem they're facing, and spending five minutes trying to come up with a creative solution. For example, talking about the CPU utilization of a web service, your service can…

Yep. For example, I use control theory to keep my services at just single percents below maximum throughput achievable on the server. Then I use other tricks (like batching processing) to make the application MORE efficient as the traffic increases.

The end effect is I can just back off 1-5% off the maximum throughput and keep the service there running happily.

I would like to use this occasion to point out that all the discussion about unused CPU is at this time completely pointless.

Most services I have seen waste ORDERS of magnitude by being inefficient. Rather than focusing on trying to saturate the CPU and other resources it is almost always better to just make your application more efficient. That last 30% should be a cherry on top.

Post reply on HN