Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

So you want to build your own data center

blog.railway.com

Post reply on HN
Print view View original

101–110 of 287 posts

Re: So you want to build your own data center

#101

Post by latchkey »

macintux wrote: post #19
jonatron wrote: post #16

Why would you call colocation "building your own data center"? You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean?

Dealing with power at that scale, arranging your own ISPs, seems a bit beyond your normal colocation project, but I haven’t bee in the data center space in a very long time.

One of the many reasons we went with Switch for our DC is because they have a service to handle all of that for you. Having stumbled on doing this ourselves before, it can be pretty tricky to negotiate everything.

We had one provider give us a great price and then bait and switch at the last moment to tell us that there is some other massive installation charge that they didn't realize we had to pay.

Switch Connect/Core is based off the old Enron business that Rob (CEO) bought...

https://www.switch.com/switch-connect/ https://www.switch.com/the-core-cooperative/

Re: So you want to build your own data center

#102

Post by Over2Chars »

chatmasta wrote: post #51
jonatron wrote: post #16

Why would you call colocation "building your own data center"? You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean?

It seems a bit disingenuous but it’s common practice. Even the hyperscalers, who do have their own datacenters, include their colocation servers in the term “datacenter.” Good luck finding the actual, physical location of a server in GCP europe-west2-a (“London”). Maybe it’s in a real Google datacenter in London! Or it could be in an Equinix datacenter in Slough, one room away from AWS eu-west-1. Cloudflare has also…

Indeed, I've seen "data center" maps, and was surprised they were just a tenant in some other guys data center.

Re: So you want to build your own data center

#103

Post by latchkey »

toddmorey wrote: post #44

Reminds me of the old Rackspace days! Boy we had some war stories: - Some EMC guys came to install a storage device for us to test... and tripped over each other and knocked out an entire Rack of servers like a comedy skit. (They uh... didn't win the contract.) - Some poor guy driving a truck had a heart attack and the crash took our DFW datecenter offline. (There were ballards to prevent this sort of scenario, but t…

We had a bird land on a transformer up on a pole and blew fuses. A couple years later, I toured the facility and the fried carcass was still there on the ground below it.

Re: So you want to build your own data center

#104

Post by vel0city »

jonatron wrote: post #16

Why would you call colocation "building your own data center"? You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean?

How to build a house:

Step 1: sign a lease at an apartment

Re: So you want to build your own data center

#107

Post by whalesalad »

jdoss wrote: post #30

This is a pretty decent write up. One thing that comes to mind is why would you write your own internal tooling for managing a rack when Netbox exists? Netbox is fantastic and I wish I had this back in the mid 2000s when I was managing 50+ racks. https://github.com/netbox-community/netbox

Netbox is just 10,000 Django models with a theme on top. Not very rewarding software to use.

Re: So you want to build your own data center

#108

Post by VTimofeenko »

justjake wrote: post #82
VTimofeenko wrote: post #73

Earlier quoted context omitted.

> We maintain Nixpacks I _knew_ Railway sounded familiar. Out of curiosity: is nix used to deploy the servers?

Not ATM. We use it in a lot of our stack, so we will likely pull it in in the future

Got it. Especially interested to see how you set up PXE. Seen a few materials out there but never got around to doing it in my lab.

Looking forward to more blogposts!

Re: So you want to build your own data center

#109

Post by ChuckMcM »

From the post: "...but also despite multi-million dollar annual spend, we get about as much support from them as you would spending $100." -- Ouch! That is a pretty huge problem for Google.

I really enjoyed this post, mostly because we had similar adventures when setting up the infrastructure for Blekko. For Blekko, a company that had a lot of "east west" network traffic (that is traffic that goes between racks vs to/from the Internet at large) having physically colocated services without competing with other servers for bandwidth was both essential and much more cost effective than paying for this special case at SoftLayer (IBM's captive cloud).

There are some really cool companies that will build an enclosure for your cold isle, basically it ensures all the air coming out of the floor goes into the back of your servers and not anywhere else. It also keeps not cold air from being entrained from the sides into your servers.

The calculations for HVAC 'CRAC' capacity in a data center are interesting too. In the first CoLo facility we had a 'ROFOR' (right of first refusal) on expanding into the floor area next to our cage, but when it came time to expand the facility had no more cooling capacity left so it was meaningless.

Once you've done this exercise, looking at the 0xide solution will make a lot more sense to you.

Re: So you want to build your own data center

#110

Post by ca508 »

whalesalad wrote: post #105

Cliffhanger! Was mostly excited about the networking/hypervisor setup. Curious to see the next post about the software defined networking. Had not heard of FRR or SONIC previously.

the good news on this is that we've got a tonne of deep-dive material on networking and whitebox switches we cut from this post. We'll definitely be talking more about this soon (also cos' BGP is cool).
Post reply on HN