Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Efficiency trades off against resiliency

blog.nelhage.com

Post reply on HN
Print view View original

41–50 of 66 posts

Re: Efficiency trades off against resiliency

#41

Post by pclmulqdq »

anonymoushn wrote: post #24
akvadrako wrote: post #23

Earlier quoted context omitted.

How would you know where an integer in JSON ends without looking ahead? You won't even know if it is an integer or not. It seems like a more expensive operation than protobuf.

You use about 2 SIMD instructions to compare 16 or 32 or 64 characters to whitespace, commas, and closing brackets, and one of these will be the character after the end of the integer literal. Theres no dependency between decoding the integer and tokenizing or decoding the next thing.

You can also use SIMD to parse varints (used by protobuf format).

Re: Efficiency trades off against resiliency

#42

Post by silvestrov »

HALtheWise wrote: post #7

While I generally agree that the pattern this article describes is real (there's some degree of tradeoff between robustness and efficiency), I've frequently seen engineers fall back on that logic rather than actually thinking about the specific problem they're facing, and spending five minutes trying to come up with a creative solution. For example, talking about the CPU utilization of a web service, your service can…

Beware of I/O, memory usage and cache eviction.

A low-priority background job can issue I/O requests that result in high I/O latencies for everything else (by e.g. making lot of random seeks).

This is very easy to get when using spinning rust.

Re: Efficiency trades off against resiliency

#43

Post by bullen »

The only interesting challenge is non-trivially parallelizable multicore applications that need to share memory across threads.

And the only choice you need to make is arrays of 64 byte structures C OR Java.

In other words do you need to be able to sleep after a day of launching new feaures?

The way I tend to do these things is to prototype with Java and once the concept/protocol is finalized and fossilized; rewrite it in C atleast on the client.

Java is the best language for server platforms, with classloader hot-deployment for sub second global turnaround and no crashes VM+GC.

About services being infinitely accessible: companies that have non-trivial solutions are now moving away from this with queues.

Limit the number of concurrent users and when saturated customers have to wait.

Open-source/source-available is the best way to scale though. If people are prepared to pay, others will run you service.

You need to allow others to make money though. I recommend revenue scaled monthly licence fees recurring on something like gumroad.

Finally I'm pretty sure the new 4nm chips will proove to be more fragile than the 14nm stack, only alot of 100% CPU time will tell.

Re: Efficiency trades off against resiliency

#44

Post by remram »

threeseed wrote: post #20
HALtheWise wrote: post #7

While I generally agree that the pattern this article describes is real (there's some degree of tradeoff between robustness and efficiency), I've frequently seen engineers fall back on that logic rather than actually thinking about the specific problem they're facing, and spending five minutes trying to come up with a creative solution. For example, talking about the CPU utilization of a web service, your service can…

> it would be really nice if load balancers used realtime performance metrics to balance traffic at a millisecond level With Kubernetes this is trivial. Horizontal Pod Autoscaling integrates with Prometheus so can spin up a new instance of your service based on whatever custom metrics you like. And it is proven to work without requiring some "creative solution" that will be buggier, less secure and inevitably less ma…

Prometheus+Kubernetes needs in the order of 30-60s to scale up, not a good match with GP's "millisecond level" ask.

(scrape interval is 15s, pod creation requires pulling images)

Re: Efficiency trades off against resiliency

#45

Post by hsn915 »

The articles starts out with the premise absolutely backwards on its head: efficiency is the exact opposite of constant 100% CPU utilization.

The more efficient the program is, the _less_ CPU resources it will consume to achieve its task.

Specially when that task is serving http requests. Unless you are always serving one million concurrent connections with one servers, you should never see chronic 100% CPU utilization.

Re: Efficiency trades off against resiliency

#46

Post by bumby »

quickthrower2 wrote: post #8
cmilton wrote: post #4

Earlier quoted context omitted.

How is peak load defined in a hospital? (I assume you mean patients. I apologize if that is incorrect.)

For a public health system you run the stats, using the $ cost of a life (or equivalent for a well life vs. sick life). Sounds sinister but ultimately they have so many $ and need to decide how to optimise that money.

>optimise that money

Perhaps the problem is defining a hospitals utility function as “optimizing money” rather than “optimizing a patient outcome”.

I agree that money is definitely a constraint, but I’m not sure it’s what should be optimized.

Re: Efficiency trades off against resiliency

#47

Post by bumby »

closeparen wrote: post #19
bsder wrote: post #3

Maximally efficient is minimally robust.

Catastrophic failure is pretty bad for efficiency. Over any serious time horizon, being maximally efficient means finding the optimal level of robustness, given the likelihoods and consequences of possible failures and the costs involved in preventing or mitigating them.

The problem is the incentives of the decision makers often don’t align with that time horizon.

Consider a CEO who is rewarded by quarterly outcomes rather than how healthy the company will be in two or three decades. Or a politician who proposes a short-term policy that looks like a short term win but will undermine constituents after they are long out of office.

Re: Efficiency trades off against resiliency

#48

Post by didgetmaster »

>developers or operators can use that slack to step in and handle unexpected load or resolve underlying issues before they become catastrophic or externally visible.

They can use 'slack' to pay off some technical debt that tends to build up when developers are always pushed to devote 100% of their time and effort to new features. There is often no time to go back and fix things that can bite you when the load reaches a certain level.

Re: Efficiency trades off against resiliency

#49

Post by hinkley »

threeseed wrote: post #21
HALtheWise wrote: post #18

Earlier quoted context omitted.

I was assuming that it's option 2 (you can spin up new workers within 10 days). If you're using cloud compute this is almost always true. If you're building an on-prem cluster, I'm assuming you either spin up cloud workers for the extra load, have extra servers that are fully powered off and can be booted in a few minutes, or just physically order and install new hardware. Amazon can ship you a computer in 2 days, so…

The idea that you can run an on-premise cluster and just supplement it with cloud workers is a nice idea in theory. But the reality is that this never happens. Because the two main reasons that a company would be running an on-premise cluster is (a) security and (b) cost. Both of which conflict with running a hybrid model because it is extremely complex and expensive to do so.

Reaching into on-prem hardware from the cloud is a substantial engineering effort, and one that can land you in the news if you do it wrong.

Re: Efficiency trades off against resiliency

#50

Post by bluepizza »

pclmulqdq wrote: post #40
bluepizza wrote: post #35

Earlier quoted context omitted.

Serialization/deserialization is still quite costly even with protobufs/flat buffers. This is not the last tiny bit. In some applications, I've seen it take more than 15% of the total CPU usage.

In some applications, this is true. I think 15% is pretty extreme, though, unless what you're doing is something like parsing/ creating data feeds (where the serialization is the point). In those cases, it's probably a good idea to have your own format. Still, JSON would be a lot worse for these cases than protobufs or flatbuffers. Also, are you sure you aren't compressing them if you're using 15% CPU?

Yeah, you almost got it :) It's a simple backing store for a few jobs that create data feeds.

Agreed that JSON would be much worse. We have a hard requirement that all services must be Java based, and protobuf outperforms Gson and Jackson by miles.

I have been trying to find the cheapest serialization/deserialization I could find for cache purposes. So far, the best option is a Guava/Caffeine cache, because we skip the serialization completely in this scenario, but so much more costly than having a good external cache.

Post reply on HN