Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

So you want to build your own data center

blog.railway.com

Post reply on HN
Print view View original

181–190 of 287 posts

Re: So you want to build your own data center

#181

Post by ksec »

I am just fascinated by the need of Datacenter. The scale is beyond comprehension. 10 years ago, before the word HyperScaler was even invented or popularised, I would have thought DC market to be on the decline or levelled off now or around this time. One reason being HyperScaler, AWS, Google, Microsoft, Meta, Apple, Tencent, Alibaba, to smaller ones like Oracle and IBM. They would all have their own DC, taking on much of the compute for themselves and others. While left over space would be occupied by third parties. Another reason being the compute, memory and storage density continue to increase, which means for the same amount of floor space we are offering 5 - 20x of the previous CPU / RAM / Storage.

Turns out we are building like mad and we are still not building enough.

Re: So you want to build your own data center

#182

Post by inopinatus »

toddmorey wrote: post #44

Reminds me of the old Rackspace days! Boy we had some war stories: - Some EMC guys came to install a storage device for us to test... and tripped over each other and knocked out an entire Rack of servers like a comedy skit. (They uh... didn't win the contract.) - Some poor guy driving a truck had a heart attack and the crash took our DFW datecenter offline. (There were ballards to prevent this sort of scenario, but t…

Many years ago I had a BlackDiamond dropped on my foot during installation at INTX LON1 for LINX, disabling me for hours. The switch in question was evidently cursed: later that week a spanning tree misconfiguration on the same unit then disabled LINX for hours, throwing half of Britain's ISP peering into temporary chaos, and everyone else involved in that project was dead within two years.

Re: So you want to build your own data center

#183

Post by inopinatus »

kryogen1c wrote: post #121
toddmorey wrote: post #44

Reminds me of the old Rackspace days! Boy we had some war stories: - Some EMC guys came to install a storage device for us to test... and tripped over each other and knocked out an entire Rack of servers like a comedy skit. (They uh... didn't win the contract.) - Some poor guy driving a truck had a heart attack and the crash took our DFW datecenter offline. (There were ballards to prevent this sort of scenario, but t…

> Data center science has... well improved since the earlier days You say that, but... > There was one day we knocked out windows and purchased box fans because servers were literally catching on fire This happened to Equinix's CH1 datacenter in Chicago Jan24 (not the literal fire part). Took down Azure ExpressRoute. Apparently it got too cold and the CRACs couldn't take it? I'm told they had all the doors and window…

running European ISPs in summer we’d nick desk fans off the telco folks to cool down our walls of USR Sportsters, distracting them first with snarky remarks about ATM overhead

absolutely do not miss those days

Re: So you want to build your own data center

#184

Post by quickthrowman »

manquer wrote: post #111
xiconfjs wrote: post #31

Earlier quoted context omitted.

I have to second this. While it takes mich effort and in-depth knowledge do build up from an “empty” cage it’s still far from dealing with everything from building permits, to plan and realize a data center to code including redundant power lines, AC and fibre. Still kudos going this path in the cloud-centric time we live in.

While it is more complex to actually build out the center , a lot of that is specific to the regional you are doing it. Thy will vary by country, by state or even county , setting up a DC in the Bay Area and say one in Ohio or Utah is a very different endeavor with different design considerations.

> Thy will vary by country, by state or even county , setting up a DC in the Bay Area and say one in Ohio or Utah is a very different endeavor with different design considerations.

Regarding data centers that cost 9 figures and up:

For the largest players, there’s not a ton of variation. A combination of evaporative cooling towers and chillers are used to reject heat. This is a consequence of evaporative open loop cooling being 2-3x more efficient than a closed-loop system.

There will be multiple medium-voltage electrical services, usually from different utilities or substations, with backup generators and UPSes and paralleling switchgear to handle failover between normal, emergency, and critical power sources.

There’s not a lot of variation since the two main needs of a data center are reliable electricity and the ability to remove heat from the space, and those are well-solved problems in mature engineering disciplines (ME and EE). The huge players are plopping these all across the country and repeatability/reliability is more important than tailoring the build to the local climate.

FWIW my employer has done billions of dollars of data center construction work for some of the largest tech companies (members of Mag7) and I’ve reviewed construction plans for multiple data centers.

Re: So you want to build your own data center

#185

Post by fishstock25 »

kqr wrote: post #176
fishstock25 wrote: post #147

Earlier quoted context omitted.

When you invite a girl/guy over, do you say "let's meet at my place" or "let's meet at the place I'm renting"? The possessive pronoun does not necessarily express ownership, it can just as well express occupancy.

I wouldn't oppose telling a client "we can meet at your data centre". I would not tell my wife "we need to discuss building our apartment complex" when we are planning interior decorations in our flat.

Haha fair enough

Re: So you want to build your own data center

#186

Post by [deleted] »

dkersten wrote: post #164
inopinatus wrote: post #133

Earlier quoted context omitted.

Come to my office and tell me how it’s not actually my office because it’s leased by my company from the investment vehicle for institutional investors that owns the building that stands on land owned by someone else again that was stolen by the British anyway and therefore calling it “my office” makes me a fool and a liar and I should just “say what I mean”.

But if I said I’m building an office, would you assume I’m furnishing an empty rented space, or constructing the building?

[deleted]

Re: So you want to build your own data center

#187

Post by aetimmes »

inopinatus wrote: post #133
jonatron wrote: post #16

Why would you call colocation "building your own data center"? You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean?

Come to my office and tell me how it’s not actually my office because it’s leased by my company from the investment vehicle for institutional investors that owns the building that stands on land owned by someone else again that was stolen by the British anyway and therefore calling it “my office” makes me a fool and a liar and I should just “say what I mean”.

I think the word GP is objecting to isn't "your own" but rather "build".

For people who have taken empty lots and constructed new data centers (ie, the whole building) on them from scratch, the phrase "building a datacenter" involves a nonzero amount of concrete.

OP seems to have built out a data hall - which is still a cool thing in its own right! - but for someone like me who's interested in "baking an apple pie from scratch", the mismatch between the title and the content was slightly disappointing.

Re: So you want to build your own data center

#188

Post by quickthrowman »

TacticalCoder wrote: post #42
jonatron wrote: post #16

Why would you call colocation "building your own data center"? You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean?

> You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean? TFA explain what they're doing, they literally write this: "In general you have three main choices: Greenfield buildout (...), Cage Colocation (getting a private space inside a provider's datacenter enclosed by mesh walls), or Rack colocation... We chose the second option" I don't kn…

Only one of those options is ‘building your own data center’, and I’ll give you three guesses as to which one it is. I’ll even give you a hint: ‘greenfield’ is in the correct answer.

Re: So you want to build your own data center

#190

Post by blmt »

I am really thankful for this article as I finally get where my coworkers get "wrong" notions about three-phase power use in DC:

>The calculations aren’t as simple as summing watts though, especially with 3-phase feeds — Cloudflare has a great blogpost covering this topic.

What's written in the Cloudflare blogpost linked in the article holds true only of you can use a Delta config (as done in the US to obtain 208V) as opposed to the Wye config used in Europe. The latter does not give a substantial advantage: no sqrt(3) boost to power distribution efficiency and you end up adding Watts for three independent single phase circuits (cfr. https://en.m.wikipedia.org/wiki/Three-phase_electric_power).

Post reply on HN