weird to think my final internship was running on one of these things. thanks for all the free minutes! it was a nice experience
So you want to build your own data center
11–20 of 287 posts
Re: So you want to build your own data center
#12First time checking out railway product- it seems like a “low code” and visual way to define and operate infrastructure?
Like, if Terraform had a nice UI?
Re: So you want to build your own data center
#13First time checking out railway product- it seems like a “low code” and visual way to define and operate infrastructure? Like, if Terraform had a nice UI?
Kinda. It's like if you had everything from an infra stack but didn't need to manage it (Kubernetes for resilience, Argo for rollouts, Terraform for safely evolving infrastructure, DataDog for observability)
If you've heard of serverless, this is one step farther; infraless
Give us your code, we will spin it up, keep it up, automate rollouts service discovery, cluster scaling, monitoring, etc
Re: So you want to build your own data center
#14Earlier quoted context omitted.
Everything is dual redundancy. We run RAID so if a drive fails it's fine; alerting will page oncall which will trigger remote hands onsite, where we have spares for everything in each datacenter
How much additional overhead is there for managing the bare-metal vs cloud? Is it mostly fine after the big effort for initial setup?
We built some internal tooling to help manage the hosts. Once a host is onboarded onto it, it's a few button clicks on an internal dashboard to provision a QEMU VM. We made a custom ansible inventory plugin so we can manage these VMs the same as we do machines on GCP.
The host runs a custom daemon that programs FRR (an OSS routing stack), so that it advertises addresses assigned to a VM to the rest of the cluster via BGP. So zero config of network switches, etc... required after initial setup.
We'll blog about this system at some point in the coming months.
Re: So you want to build your own data center
#15Can anyone recommend some engineering reading for building and running DC infrastructure?
Re: So you want to build your own data center
#16Why would you call colocation "building your own data center"? You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean?
Re: So you want to build your own data center
#17Love these kinds of posts. Tried railway for the first time a few days ago. It was a delightful experience. Great work!
Re: So you want to build your own data center
#18Can anyone recommend some engineering reading for building and running DC infrastructure?
We didn't find many good up-to-date resources online on the hardware side of things - kinda why we wanted to write about it. The networking aspect was the most mystical - I highly recommend "BGP in the datacenter" by Dinesh Dutt on that (I think it's available for free via NVidia). Our design is heavily influenced by the ideas discussed there.
Re: So you want to build your own data center
#19Why would you call colocation "building your own data center"? You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean?
Dealing with power at that scale, arranging your own ISPs, seems a bit beyond your normal colocation project, but I haven’t bee in the data center space in a very long time.
Re: So you want to build your own data center
#20Awesome!! Hope to see more companies go this route. I had the pleasure to do something similar for a company(lot smaller scale though)
It was my first job out of university. I will never forget the awesome experience of walking into the datacenter and start plugging cables and stuff