Live data from Hacker News

Facebook open sources its servers and data centers

gigaom.com

81–90 of 105 posts

Re: Facebook open sources its servers and data centers

#81
post #29

say what you want about Facebook but I give them props for open sourcing so much code to the community. Cassandra, Thrift, Scribe,Hive, etc...

"Please take a look, tell us what we did wrong and join us in working together to make every data center more efficient." Mad Props.

Indeed. This is how I want to see society. Sharing knowledge for the greater good. Probably the reason I'm not a fan of detriments of this such as software patents and Apple in general.

Re: Facebook open sources its servers and data centers

#82
post #28

I don't get the AC PSUs on the servers ( http://opencompute.org/servers/ ). Any reason why those are being used when each 6-rack group has a UPS (and batteries) connected to them? Going DC would get rid of the inverter on the UPS out and the 200 or so PSUs on each server.

I'm also puzzled why they don't go all-DC. This thread doesn't appear to be getting any traction here. Is there a HN for datacenter geeks?

Re: Facebook open sources its servers and data centers

#83
post #28

I don't get the AC PSUs on the servers ( http://opencompute.org/servers/ ). Any reason why those are being used when each 6-rack group has a UPS (and batteries) connected to them? Going DC would get rid of the inverter on the UPS out and the 200 or so PSUs on each server.

Amir Michael who worked on the hardware team said, "The utility provides us with AC voltage which we then convert to DC very close to the motherboard in the power supply. Our goal was to bring the high voltage as close as possible to the load to minimize IR losses. We could do an efficient AC –DC conversion in the rack and then distribute DC to the individual servers but that would mean several feet of low voltage conductors which would be made from large copper bars and a would have higher IR losses."

Re: Facebook open sources its servers and data centers

#84
post #70

Earlier quoted context omitted.

Becuase Search is ultimately about technology, but Social is about the users. If I was the only user of Google (and they had the same technology stack) then +1 aside - my experience would be the same as it is now. If I was the only user of facebook - what would be the point.

I'm sorry, but it seems like you're implying that search doesn't need users. Some of Google's ranking depends on analyzing search histories, so if you were the only user, the experience would be worse. That means that search is also about the users. Data center technology can benefit everybody, that's all I'm trying to say.

>Google's ranking depends on analyzing search histories, so if you were the only user, the experience would be worse.

I disagree. At this point, I believe that because of how big google is and because of how profitable gaming google is, getting good results, I believe that more than half of the work of finding relevant search results is filtering out the people attempting to game the search engine.

I mean, it is /possible/ that the help they get from user's search histories outweighs the massive amount of effort spammers put in to making google suck, but I doubt it.

I bet that things are much easier for a new search engine while they are small enough that it's not worth much time and effort to game the results.

>Data center technology can benefit everybody, that's all I'm trying to say.

This is true, but I believe what other users are likely saying is that google probably alrealdy has something as good or better than this.

Personally, I won't get excited until someone starts building out co-location centers that have efficent cooling. Nearly all co-location centers use mixed air, and most end up keeping the place much cooler than required, and they pass on those costs to me.

The thing is, for anyone who is big enough to take advantage of their data center design, this isn't particularly new. I mean, it's great they are sharing the documentation of how they did it... but that mostly helps the little guys, and the little guys can't really take advantage of it... we're stuck in co-location centers, like I said, with antiquated cooling designs.

The chassis designs, on the other hand, might be great for the little guys. Hell, I am not that far away from enough scale to job out the chassis to a metal shop, and the design would be a pretty big part of that cost, that facebook just reduced to zero. Thanks!

But the power savings on the chassis are very small compared to the power savings on the cooling/data center design.

Re: Facebook open sources its servers and data centers

#85
post #35
post #12

Earlier quoted context omitted.

Okay, but why does it benefit Facebook's bottom line (and Google's!) to open-source that stuff? Wouldn't they be better off if their competitors had to pay those higher costs? It's not as if they have substantial computation-service providers they're buying from, or data-center-oriented companies they're selling to.

It's not about the fact that open sourcing this info affects their bottom line as much as it is about responsibly contributing to the progress and advancement of computing... however it should be noted that for years, google has been very secretive about all aspects of its internal DC designs. The fact they were making their own equipment, incorporating UPS functions in the power supplies of their servers, how they w…

You can buy the lighting yourself, it's from Redwood Systems: http://www.redwoodsystems.com/products

Re: Facebook open sources its servers and data centers

#86

I'd be very interested in hearing from someone with more experience in running their own hardware what portions (if any) of what Facebook has announced today is applicable at the small scale of say having a couple of co-located racks in some datacenter. Maybe the base server designs?

I can give you an interesting data point: our power bill is only about $600/mo (2.5KWh), our bandwidth bill is about $3,500/mo (1.2Gbps). So power is a smaller consideration.

Re: Facebook open sources its servers and data centers

#87
post #42
post #34

Earlier quoted context omitted.

Any idea on how much would it cost to produce a small quantity of those boards? At which scale does it starts to make sense to use a custom made motherboard instead of a off-the-shelf PC?

One thing to keep in mind is that one rack holds 90 servers and each battery rack feeds 180 servers, so (at least in Facebook's case) that's the basic unit. If this stuff takes off I would expect someone to produce a smaller 30-slot rack, though.

No. Each rack holds 30 servers and each triplet holds 90. The battery cabinet can support 2 triplets.

http://opencompute.org/datacenters/

Re: Facebook open sources its servers and data centers

#88
post #60

This is a strategic attack on Google. A proliferation of scalable data centers hurts Google a lot more than Facebook by enabling Google's competitors. Cheap computation matters much more to search engines than social networking sites.

That seems like an unlikely reason. If you are looking for a reason beyond altruism, they hope to gain from improvements others make to the platform they are already invested in using. For example, if a hardware manufacturer sees this and then chooses to build some hardware controller that works a little better with Facebook's setup, Facebook wins.

Re: Facebook open sources its servers and data centers

#89
post #57

Earlier quoted context omitted.

Pretty standard in telecoms. Never quite understood why Google went per server.

Their claims were that the efficiency was that great, and that it's acceptable to pay the extra few dollars in motherboard manufacture to build it in.

I guess in their instance the lifetime of the server was never likely to be eclipsed by that of the battery. In telco, 15-20yrs of operation is not uncommon, so a single set of replaceable/easily service batteries is important.

Re: Facebook open sources its servers and data centers

#90

Earlier quoted context omitted.

I am curious if they would ever look into loongson. It's 64bit and fairly power efficient.

There are other MIPS licensees too.

True, but loongson is the first one that comes to my mind. I'm not familiar with the other MIPS licensees or the power consumption of their chips.
Post reply on HN