Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why the data center needs an operating system

radar.oreilly.com

Post reply on HN
Print view View original

51–60 of 102 posts

Re: Why the data center needs an operating system

#51

Post by derefr »

davidgerard wrote: post #34
preillyme wrote: post #30

Earlier quoted context omitted.

ha ha ha

ha ha only serious http://www.catb.org/jargon/html/H/ha-ha-only-serious.html It's VAXocentrism for the 21st century. http://www.catb.org/jargon/html/V/vaxocentrism.html

Interesting to think that the assumptions of C as a language evolved mostly against the VAX architecture. It seems that if the VAX doesn't make a distinction, then C doesn't (tend to) have any concept of that distinction either. Examples:

- Pointer types are basically fungible in C (otherwise there would be no void-ptr type)

- There's no compile-time knowledge of the segment a pointer references to prevent you from dereferencing a pointer to an offset from segment A when segment B is loaded (compare this to Rust's parameterization of Box types by their allocator)

- Struct padding is painful and tacked on

- "unsigned char" isn't default even though it'd make much more sense for it to be (What "char"acter is negative? You can have a signed byte/octet, but a character is—in 1979, at least—basically an enum/sum type.

Re: Why the data center needs an operating system

#52

Post by stevedomin »

chuhnk wrote: post #26

It's interesting and great to see this stuff take center stage more and more. Those lucky enough to work at places like Twitter, Google, Facebook and other large tech companies will have already seen how this kind of thing dominates the datacenter's there and has been at the core of their systems for many years. People on the outside though are rarely exposed to this concept of datacenter scale computing aside from t…

I agree with you but I don't think we'll have to wait 10 years to see that happening. We're still a fairly small startup but we're transitioning our whole platform to be built around Mesos (and we're definitely not the only ones in that case).

I don't know why many people think that they need to be at datacenter scale computing to benefit from abstractions like Mesos, it's completely wrong imo.

It's quite a big shift in mindset but it makes the life of everyone (dev and ops) so much easier when you stop having to think about single machines.

Re: Why the data center needs an operating system

#54

Post by skj »

mattzito wrote: post #47
chuhnk wrote: post #26

It's interesting and great to see this stuff take center stage more and more. Those lucky enough to work at places like Twitter, Google, Facebook and other large tech companies will have already seen how this kind of thing dominates the datacenter's there and has been at the core of their systems for many years. People on the outside though are rarely exposed to this concept of datacenter scale computing aside from t…

> Google SREs by last count were 1 engineer to 1000 machines That number does not seem particularly impressive, if it is accurate. Even "traditional" well-run enterprise IT organizations are often in the 1 admin/SRE to 600-ish machines, so I have a hard time seeing that Google can only do ~2x as good at their scale and with their level of focus. 1 SRE to 5k machines, 10k machines, that makes more sense to me.

You'll never see an accurate ratio, because revealing it would reveal the number of machines in Google data centers (considered a trade secret).

Re: Why the data center needs an operating system

#55

Post by ryanburk »

mattzito wrote: post #47
chuhnk wrote: post #26

It's interesting and great to see this stuff take center stage more and more. Those lucky enough to work at places like Twitter, Google, Facebook and other large tech companies will have already seen how this kind of thing dominates the datacenter's there and has been at the core of their systems for many years. People on the outside though are rarely exposed to this concept of datacenter scale computing aside from t…

> Google SREs by last count were 1 engineer to 1000 machines That number does not seem particularly impressive, if it is accurate. Even "traditional" well-run enterprise IT organizations are often in the 1 admin/SRE to 600-ish machines, so I have a hard time seeing that Google can only do ~2x as good at their scale and with their level of focus. 1 SRE to 5k machines, 10k machines, that makes more sense to me.

having a ratio where you scale linearly is really not common in mega-scale web services since you have infrastructure services like borg / mesos / omega / autopilot / etc.

you can take a snapshot and say you are 1:n because today you have so many sre and so many machines, but it is very unlikely to be the same ratio down the road.

Re: Why the data center needs an operating system

#56

Post by aidenn0 »

derefr wrote: post #51
davidgerard wrote: post #34

Earlier quoted context omitted.

ha ha only serious http://www.catb.org/jargon/html/H/ha-ha-only-serious.html It's VAXocentrism for the 21st century. http://www.catb.org/jargon/html/V/vaxocentrism.html

Interesting to think that the assumptions of C as a language evolved mostly against the VAX architecture. It seems that if the VAX doesn't make a distinction, then C doesn't (tend to) have any concept of that distinction either. Examples: - Pointer types are basically fungible in C (otherwise there would be no void-ptr type) - There's no compile-time knowledge of the segment a pointer references to prevent you from d…

Actually C disallows converting function pointers to/from void pointers.

Re: Why the data center needs an operating system

#58

Post by mattbee »

The article's premise is a poor introduction to the project. Sure, reinvent MOSIX if you want :) but don't pretend it'll serve more than a niche of a niche.

Firstly, "distributed computing is the norm"? It's just not. Most businesses & app authors will never need to care about ultra-distributed computing, with all its problems and trade-offs. You can move faster with "local-only" computing and scale vertically very cheaply compared to a few years ago - 4 dedicated CPUs + hundreds of gigabytes of RAM save programmer hours, and get your problem solved faster.

For light scaling issues (compared to Google) Redis, MariaDB and other abstractions over local files have some great options for future scaling, and are well-trodden, obvious choices.

Secondly, who cares about "wasted" resources of a whole underutilised server when reliable dedicated servers are so cheap, and in such plentiful supply?

Thirdly, "organizations must employ armies of people to manually configure and maintain each individual application on each individual machine"? - in the 90s maybe! Surely anyone with more than a few applications to worry about is on board with some basic configuration management.

Twitter-size scaling is a "nice problem to have". For all but the best-funded & bullish companies, solve them only when you start to have them.

(my bias: I run a managed service provider in the UK - we tend to help customers scale vertically by shovelling server images around with minimum down time. We say "underused" dedicated server capacity at fixed monthly costs is usually cheaper than chasing the phantom of "optimum" AWS usage.)

Re: Why the data center needs an operating system

#59

Post by srean »

stryan wrote: post #41

This sounds like the problem Plan 9 set out to solve

Upvoted. In fact while reading the article I was just thinking to myself that the HN comments would be all about Plan 9. Kind of surprised that it has not been mentioned enough.

What I am really keen to find out in the coming years is what MirageOS makes of this. If you are not familiar this article http://queue.acm.org/detail.cfm?id=2566628 explains it way better than I could. I wouldnt claim it is there yet but seems to be sitting right at an envious position full of realizable potential. Its written in OCaml to boot.

Re: Why the data center needs an operating system

#60

Post by toast0 »

presspot wrote: post #25
joefarish wrote: post #6

Earlier quoted context omitted.

What makes you say that?

Datacenter and back-end apps simply don't fit on a single machine anymore. Every app of reasonable scale is probably a distributed system of some sort. That, and there are a new class of "apps" (or, more precisely, datacenter services) that were built to operate across fleets of machines from day one, such as Spark, Hadoop, Cassandra, Kafka, Elasticsearch, and so on

On the other hand, a single machine can get pretty big today, for example http://www.supermicro.com/products/system/2U/6028/SYS-6028U-... is 2u, has 12 drive bays and can have 1.5 tb ram. You could do 24 bays if you are using 2.5" drives (most ssds). A lot of complexity can be avoided by getting many fewer, but bigger machines (doesn't help much if you need massive CPU though, but intels architectures are getting better every cycle)
Post reply on HN