Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Clark's Curious Corollary of Concurrency

clarkgaebel.com

Post reply on HN
Print view View original

11–20 of 23 posts

Re: Clark's Curious Corollary of Concurrency

#11

Post by scott_s »

twic wrote: post #10
scott_s wrote: post #8

Performance is hard, but we should all be working together towards the things that matter the most. Latency is more important to optimize than throughput. Let's just focus on that. I believe this is naive. First, it has no context. You optimize for what you need; there is no absolute best thing. If you have fantastic latency, but your throughput is not good enough to meet your needs, then, no, latency is not more imp…

> You optimize for what you need; there is no absolute best thing. If you have fantastic latency, but your throughput is not good enough to meet your needs, then, no, latency is not more important. The whole point of this post is that if you have low latency, there are easy ways to trade it away for better throughput. Whereas there are not easy ways to trade high throughput for better latency. And, therefore, latency…

And my point is that absent knowing what your needs are, it's silly to talk about what is "more important". Because of the lopsided nature of the tradeoff (small latency harm for big throughput gain), it's dangerous to keep around "latency is more important" as a mantra.

Re: Clark's Curious Corollary of Concurrency

#12

Post by marcosdumay »

neilc wrote: post #7

Another factor is that technology improvements tend to favor increases in bandwidth over increases in latency (and there are hard limitations on latency, like the speed of light in distributed systems). This short paper by David Patterson is a great read on the subject: http://www.cis.gvsu.edu/~kurmasz/Teaching/OldCourses/CS451/G...

Well, in practice the latency that normaly matters is between "start doing activity" and "finish doing activity", and the time actualy working (that depends on bandwidth) is normaly orders of magnitude bigger than the time waiting for data (that depends on latency). There are exceptions, but not many.

And now that I really thought about it, looks like the article's law isn't that relevant in practice. Yes, you can always trade latency for bandwidth if you throw some money at it. But money is finite.

Re: Clark's Curious Corollary of Concurrency

#13

Post by mcguire »

scott_s wrote: post #8

Performance is hard, but we should all be working together towards the things that matter the most. Latency is more important to optimize than throughput. Let's just focus on that. I believe this is naive. First, it has no context. You optimize for what you need; there is no absolute best thing. If you have fantastic latency, but your throughput is not good enough to meet your needs, then, no, latency is not more imp…

"But it's always either reversing one of the tricks above, or exploiting some domain-specific parallelism."

That's kind of the point of a genuine tradeoff: you reverse what you did to improve one to improve the other. But when you say, "we tend to trade a small increase in latency for a large increase in throughput", you have the heart of what the author is complaining about. If you reverse a small increase of latency/large increase in throughput, you get a large decrease in throughput for only a small decrease in latency. To decrease latency, you have to do something else, making it the factor to watch.

Re: Clark's Curious Corollary of Concurrency

#14

Post by scott_s »

mcguire wrote: post #13
scott_s wrote: post #8

Performance is hard, but we should all be working together towards the things that matter the most. Latency is more important to optimize than throughput. Let's just focus on that. I believe this is naive. First, it has no context. You optimize for what you need; there is no absolute best thing. If you have fantastic latency, but your throughput is not good enough to meet your needs, then, no, latency is not more imp…

" But it's always either reversing one of the tricks above, or exploiting some domain-specific parallelism. " That's kind of the point of a genuine tradeoff: you reverse what you did to improve one to improve the other. But when you say, "we tend to trade a small increase in latency for a large increase in throughput", you have the heart of what the author is complaining about. If you reverse a small increase of late…

Just because it's harder, doesn't mean it's more important. That's my complaint: the author has jumped from an observation to a value judgement.

Re: Clark's Curious Corollary of Concurrency

#15

Post by tbrownaw »

This only works for "embarrassingly parallel" tasks, where you have lots of completely independent things to do. Please remember that the whole world is not a web server.

This is not trading latency for thruput. This is spending hardware for thruput, with some overhead in latency.

The relevant generic performance number is CPU-seconds (for CPU-bound work), or I/O consumption (for disk-bound work), or in general how much of your bottleneck resource is consumed. Once you know your bottleneck, you can either improve your code to use less of that resource, or buy more of that resource.

As tasks become less embarrassingly parallel, throwing (non-serial) hardware at a problem increases communication overhead, and gives lower speedups.

Re: Clark's Curious Corollary of Concurrency

#16

Post by finnh »

"Tell me the 99th percentile latency. Track it. Improve it. When needed, throughput comes easy."

Actually, if you're worried about tail latency, it can be improved via throughput. Expensively:

Double your entire infrastructure. Send each request twice, once to each half. Take the results of the faster one.

I assume that Google does something like this for fast search results.

Re: Clark's Curious Corollary of Concurrency

#17

Post by neilc »

marcosdumay wrote: post #12
neilc wrote: post #7

Another factor is that technology improvements tend to favor increases in bandwidth over increases in latency (and there are hard limitations on latency, like the speed of light in distributed systems). This short paper by David Patterson is a great read on the subject: http://www.cis.gvsu.edu/~kurmasz/Teaching/OldCourses/CS451/G...

Well, in practice the latency that normaly matters is between "start doing activity" and "finish doing activity", and the time actualy working (that depends on bandwidth) is normaly orders of magnitude bigger than the time waiting for data (that depends on latency). There are exceptions, but not many. And now that I really thought about it, looks like the article's law isn't that relevant in practice. Yes, you can al…

Well, in practice the latency that normaly matters is between "start doing activity" and "finish doing activity", and the time actualy working (that depends on bandwidth) is normaly orders of magnitude bigger than the time waiting for data (that depends on latency).

I don't think that's true; moreover, it is likely to become even less true in the future. Once the data needed for a computation has arrived at the CPU, for most applications the required computation is pretty cheap -- the time spent waiting for the data often dominates the time spent computing with the data. Much of modern CPU design has been devoted to trying to hide the high latency of memory accesses.

Re: Clark's Curious Corollary of Concurrency

#18

Post by packetslave »

finnh wrote: post #16

"Tell me the 99th percentile latency. Track it. Improve it. When needed, throughput comes easy." Actually, if you're worried about tail latency, it can be improved via throughput. Expensively: Double your entire infrastructure. Send each request twice, once to each half. Take the results of the faster one. I assume that Google does something like this for fast search results.

Yep, and you don't even (necessarily) need to double your infrastructure to do it. Jeff Dean talks about some strategies in "The Tail at Scale"

http://research.google.com/pubs/pub40801.html (paper)

http://www.youtube.com/watch?v=C_PxVdQmfpk (talk)

Re: Clark's Curious Corollary of Concurrency

#19

Post by NathanKP »

@cgaebel The code for your blog seems to have a race condition in it where the content is all there on the page, its just invisible, and only appears properly about 1 in 10 times. Either that or maybe your code is just hanging because I'm not allowing it to load Google Analytics like it wants.

http://i.imgur.com/w1ZB7Qy.png

I'm using Google Chrome 33.0.1750.149 on Mac OS X. I had to open dev tools and tweak your CSS to get the invisible text to appear so I could read it.

Re: Clark's Curious Corollary of Concurrency

#20

Post by finnh »

ArbitraryLimits wrote: post #5

http://rescomp.stanford.edu/~cheshire/rants/Latency.html > Years ago David Cheriton at Stanford taught me something that seemed very obvious at the time -- that if you have a network link with low bandwidth then it's an easy matter of putting several in parallel to make a combined link with higher bandwidth, but if you have a network link with bad latency then no amount of money can turn any number of them into a lin…

Theoretically, if your bandwidth is high enough, you can transfer the entire computational state of the distant resource to a local substrate, and then run the computation locally for a low-latency conversation.

So, if you are annoyed by the slow comms of our alpha centauri - earth channel, just transfer _your entire brain_ to a local avatar and I'll converse with that. Then run "git merge" to bring the remote history back to the master repo.

Post reply on HN