Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The C10M problem

c10m.robertgraham.com

Post reply on HN
Print view View original

61–70 of 116 posts

Re: The C10M problem

#61

Post by slashnull »

ganessh wrote: post #44

"There is no way for the primary service (such as a web server) to get priority on the system, leaving everything else (like the SSH console) as a secondary priority" - Can't we use the nice command (nice +n command) when these process are started to change its priority? I am sorry if it is so naive question

He probably meant that from a TCP point of view, as in there is no way to give a higher priority to incoming TCP connections going into the server than to those going into SSH, even if you could use nice to assign more cpu time to your server.

Or perhaps he meant that the infrastructure used to do multitasking still have to interrupt both his server and SSH, but then described how the kernel can be set to leave some cores free of work then to set the server to use only those and then run absolutely uninterrupted.

Not the only bizarre and confusing statement he wrote, anyways.

Re: The C10M problem

#62

Post by dschiptsov »

So, he is trying to suggest that pthread-mutex based approach won't scale (what a news!) and, consequently JVM is crap after all?) The next step would be to admit that the very idea to "parallelize" sequential code which imperatively processes sequential data by merely wrapping it into threads is, a nonsense too?) Where this world is heading to?

Re: The C10M problem

#63

Post by logicchains »

memracom wrote: post #48

Just what are these resources that we are using more efficiently? CPU? RAM? Are they that important? Should we not be trying to use electricity more efficiently since that is a real world consumable resource. How many connections can you handle per kilowatt hour?

Generally electricity use is roughly proportional to CPU and RAM usage, as they're powered by electricity. If you have two otherwise identical programs, one of which uses 50% of the cpu and the other of which use 10%, chances are the latter will use less electricity.

Re: The C10M problem

#64

Post by logicchains »

sitkack wrote: post #59
srean wrote: post #42

Earlier quoted context omitted.

I think the "just good enough for business" attitude contributed to the demise of the American car and the rise of the Japanese ones. The American tradition was to use engineering tolerances that would maximize throughput under the constraint that it produced a pretty functional car. The Japanese tradition on the other hand was to use tighter tolerances, well because there was room for tightening the tolerance. At th…

Your recollection of Japanese manufacturing is clouded by mythology. Japanese quality sucked in the 1950s, they had a horrible world wide reputation for shoddy goods.

Maybe German manufacturing would be a better example?

Re: The C10M problem

#65

Post by justincormack »

wpietri wrote: post #13

On the one hand, I love this. There's an old-school, down-to-the-metal, efficiency-is-everything angle that resonates deeply with me. On the other hand, I worry that just means I'm old. There are a lot of perfectly competent developers out there that have very little idea about the concerns that motivate thinking like this C10M manifesto. I sometimes wonder if my urge toward efficiency something like my grandmother's…

I think there has been a renewed interest around 10Gb ethernet (and SSD). Basically computers were grossly limited by IO with gigabit ethernet and hard drives, you could buy a pretty low end box and saturate them, instead people bought high end boxes and ran slow code. But your average slow code wont perform well on 10Gb.

Re: The C10M problem

#66

Post by tluyben2 »

dbenhur wrote: post #43
electrograv wrote: post #25

Earlier quoted context omitted.

It really just depends on how much of a computational surplus you have. Some tasks have become so incredibly easy relative to today's computational horsepower, that we can afford waste 90% of it if it means we can be 10% more productive. But this isn't true everywhere. Imagine if Google's servers cost 100x more to run, if they didn't spend time to make their code efficient. Imagine if your video games ran at a frame…

> when languages like Python and Ruby can be over 100x slower than other high level languages You said a wrong thing. Java is a far cry from a high-level language. Javascript is arguably a high-level language however, and meaningless benchmarks (aka "damn lies") attest V8 is in spitting distance of Java. http://benchmarksgame.alioth.debian.org/u32/javascript.php Scala is much higher level than Java, compiles to the s…

What makes you say Java is not a high level language? It most certainly is that according to the official definitions of the term.

Re: The C10M problem

#67

Post by jsnell »

nl wrote: post #52
erichocean wrote: post #16

What's significant to me is that you can do this stuff today on stock Linux. No need to run weird single-purpose kernels, strange hypervisors, etc. You can SSH into your box. You can debug with gdb. Valgrind. Everything is normal...except the performance, which is just insane. Given how easy it is, there isn't really a good excuse anymore to not write data plane applications the "right" way, instead of jamming everyt…

Snabb Switch looks awesome. Is there a list of "things" (not sure of the terminology?) people have built with it? I guess the downside is that you can't virtualize it (I realize that is kind of the point, but it does reduce the accessibility of it).

Some NICs support virtualizing raw hardare access, so there's no fundamental reason for why Snabb couldn't support it. Just a simple matter of programming (and last I heard it was high on the priority list, so the support might even be there by now).

Re: The C10M problem

#68

Post by jon-wood »

zerop wrote: post #41

One more problem is cloud. We host on cloud. cloud service providers might be using old hardware. Newest hardware or specific OS might be winner but no options on cloud. How do you tackle that ?

If this sort of thing matters that much to you then you bite the bullet and rent a rack to fill with physical servers somewhere.

Re: The C10M problem

#69

Post by _delirium »

nl wrote: post #52
erichocean wrote: post #16

What's significant to me is that you can do this stuff today on stock Linux. No need to run weird single-purpose kernels, strange hypervisors, etc. You can SSH into your box. You can debug with gdb. Valgrind. Everything is normal...except the performance, which is just insane. Given how easy it is, there isn't really a good excuse anymore to not write data plane applications the "right" way, instead of jamming everyt…

Snabb Switch looks awesome. Is there a list of "things" (not sure of the terminology?) people have built with it? I guess the downside is that you can't virtualize it (I realize that is kind of the point, but it does reduce the accessibility of it).

Another downside is that if you do networking in userspace, portability becomes your problem. If you do TCP/IP in the kernel, it works on nearly everything: just about any brand of Ethernet card, and even exotic non-Ethernet stuff (ISDN, whatever). If you do it in Snabb Switch, according to the wiki it currently supports exactly one class of interface: Intel-branded ethernet cards. Of course, I expect that will expand, but each one of these user-space networking stacks will have to ship with its own complete driver set to reach the same level of portability.

Re: The C10M problem

#70

Post by nostrademons »

sitkack wrote: post #59
srean wrote: post #42

Earlier quoted context omitted.

I think the "just good enough for business" attitude contributed to the demise of the American car and the rise of the Japanese ones. The American tradition was to use engineering tolerances that would maximize throughput under the constraint that it produced a pretty functional car. The Japanese tradition on the other hand was to use tighter tolerances, well because there was room for tightening the tolerance. At th…

Your recollection of Japanese manufacturing is clouded by mythology. Japanese quality sucked in the 1950s, they had a horrible world wide reputation for shoddy goods.

...which got better because Japanese companies had a culture of continuous improvement. That's very relevant to the discussion: the point the parent posters were making is that rather than settle for "good enough" you should continually move the bar for what "good enough" is, because then you'll eventually end up overtaking fat & lazy competitors even if they have a large head start.
Post reply on HN