Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

So you want to build your own data center

blog.railway.com

Post reply on HN
Print view View original

121–130 of 287 posts

Re: So you want to build your own data center

#121

Post by kryogen1c »

toddmorey wrote: post #44

Reminds me of the old Rackspace days! Boy we had some war stories: - Some EMC guys came to install a storage device for us to test... and tripped over each other and knocked out an entire Rack of servers like a comedy skit. (They uh... didn't win the contract.) - Some poor guy driving a truck had a heart attack and the crash took our DFW datecenter offline. (There were ballards to prevent this sort of scenario, but t…

> Data center science has... well improved since the earlier days

You say that, but...

> There was one day we knocked out windows and purchased box fans because servers were literally catching on fire

This happened to Equinix's CH1 datacenter in Chicago Jan24 (not the literal fire part). Took down Azure ExpressRoute.

Apparently it got too cold and the CRACs couldn't take it? I'm told they had all the doors and windows open trying to keep things cold enough, but alas. As the CRAC goes, so goes the servers

Re: So you want to build your own data center

#122

Post by motoboi »

I’m my experience and based on writeups like this: Google hates having customers.

Someone decided they have to have a public cloud, so they did it, but they want to keep clients away with a 3 meter pole.

My AWS account manager is someone I am 100% certain would roll in the mud with me if necessary. Would sleep in the floor with us if we asked in a crisis.

Our Google cloud representatives make me sad because I can see that they are even less loved and supported by Google than us. It’s sad seeing someone trying to convince their company to sell and actually do a good job providing service. It’s like they are setup to fail.

Microsoft guys are just bulletproof and excel in selling, providing a good service and squeezing all your money out of your pockets and you are mortally convinced it’s for your own good. Also have a very strange cloud… thing.

As for the railway company going metal, well, I have some 15 years of experience with it. I’ll never, NEVER, EVER return to it. It’s just not worth it. But I guess you’ll have to discover it by yourselves. This is the way.

You soon discover what in freaking world is Google having so much trouble with. Just make sure you really really love and really really want to sell service to people, instead of building borgs and artificial brains and you’ll do 100x better.

Re: So you want to build your own data center

#123

Post by mdaniel »

hintymad wrote: post #119

Per my experience with cloud, the most powerful Infra abstraction that AWS offers is actually EC2. The simplicity of getting a cluster of machines up and running with all the metadata readily available via APIs is just liberating. And it just works: the network is easy to configure, the ASGs are flexible enough to customize, and the autoscaling offers strong primitives for advanced scaling. Amazingly, few companies w…

I believe eBay still runs on OpenStack, which as far as I know even has as ec2-compatible emulation layer https://docs.openstack.org/api-ref/ec2-api/#supported-featur...

Re: So you want to build your own data center

#124

Post by mdaniel »

ca508 wrote: post #18
__fst__ wrote: post #15

Can anyone recommend some engineering reading for building and running DC infrastructure?

We didn't find many good up-to-date resources online on the hardware side of things - kinda why we wanted to write about it. The networking aspect was the most mystical - I highly recommend "BGP in the datacenter" by Dinesh Dutt on that (I think it's available for free via NVidia). Our design is heavily influenced by the ideas discussed there.

the title page says 2017 if that matters to anyone: https://docs.jetstream-cloud.org/attachments/bgp-in-the-data...

Re: So you want to build your own data center

#125

Post by greenavocado »

toddmorey wrote: post #44

Reminds me of the old Rackspace days! Boy we had some war stories: - Some EMC guys came to install a storage device for us to test... and tripped over each other and knocked out an entire Rack of servers like a comedy skit. (They uh... didn't win the contract.) - Some poor guy driving a truck had a heart attack and the crash took our DFW datecenter offline. (There were ballards to prevent this sort of scenario, but t…

When it comes to Internet service we're living in the early 2000s in the some parts of the manufacturing world

Re: So you want to build your own data center

#127

Post by ThinkBeat »

1. Is the impression they decided to use a non datacenter location to put their datacenter, If so that is not a good idea.

2. Geographical distanced backups, if the primary fails. Without this you are already in trouble. What happens if the buildings burns down?

3. Hooking up with "local" ISPs That seems ok. As long as ISP failing is easily and autoamically dealt with.

4. I am a bit confused about what happens at the edge. On the one head it seems like you have 1 datacenter, and ISPs doing routing, other places I get the impression you have compute close to the edge. Which is it?

Re: So you want to build your own data center

#128

Post by wmf »

ThinkBeat wrote: post #127

1. Is the impression they decided to use a non datacenter location to put their datacenter, If so that is not a good idea. 2. Geographical distanced backups, if the primary fails. Without this you are already in trouble. What happens if the buildings burns down? 3. Hooking up with "local" ISPs That seems ok. As long as ISP failing is easily and autoamically dealt with. 4. I am a bit confused about what happens at the…

1. No, they're using a cage inside a real data center in Ashburn VA which is basically data center city.

2. In the diagram you can see site 1 and site 2.

3. Yes, routers automatically deal with ISP failures.

Re: So you want to build your own data center

#130

Post by chatmasta »

Over2Chars wrote: post #102
chatmasta wrote: post #51

Earlier quoted context omitted.

It seems a bit disingenuous but it’s common practice. Even the hyperscalers, who do have their own datacenters, include their colocation servers in the term “datacenter.” Good luck finding the actual, physical location of a server in GCP europe-west2-a (“London”). Maybe it’s in a real Google datacenter in London! Or it could be in an Equinix datacenter in Slough, one room away from AWS eu-west-1. Cloudflare has also…

Indeed, I've seen "data center" maps, and was surprised they were just a tenant in some other guys data center.

Which makes you a subletter, and the one with the highest fee of the whole chain…
Post reply on HN