Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

So you want to build your own data center

blog.railway.com

Post reply on HN
Print view View original

31–40 of 287 posts

Re: So you want to build your own data center

#31

Post by xiconfjs »

jonatron wrote: post #16

Why would you call colocation "building your own data center"? You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean?

I have to second this. While it takes mich effort and in-depth knowledge do build up from an “empty” cage it’s still far from dealing with everything from building permits, to plan and realize a data center to code including redundant power lines, AC and fibre.

Still kudos going this path in the cloud-centric time we live in.

Re: So you want to build your own data center

#32

Post by ca508 »

jdoss wrote: post #30

This is a pretty decent write up. One thing that comes to mind is why would you write your own internal tooling for managing a rack when Netbox exists? Netbox is fantastic and I wish I had this back in the mid 2000s when I was managing 50+ racks. https://github.com/netbox-community/netbox

we evaluated a lot of commercial and oss offerings before we decided do go build it ourselves - we still have a deploy of netbox somewhere. But our custom tool (Railyard) works so well because it integrates deeply into the our full software, hardware and orchestration stack. The problem with the OSS stuff is that it's almost too generic - you shape the problem to fit its data model vs. solve the problem. We're likely going to fold our tool into Railway itself eventually - want to go on-prem; button click hardware design, commission, deploy and devex. Sorta like what Oxide is doing, but approaching the problem from the opposite side.

Re: So you want to build your own data center

#34

Post by thornewolf »

justjake wrote: post #13
nextworddev wrote: post #12

First time checking out railway product- it seems like a “low code” and visual way to define and operate infrastructure? Like, if Terraform had a nice UI?

Kinda. It's like if you had everything from an infra stack but didn't need to manage it (Kubernetes for resilience, Argo for rollouts, Terraform for safely evolving infrastructure, DataDog for observability) If you've heard of serverless, this is one step farther; infraless Give us your code, we will spin it up, keep it up, automate rollouts service discovery, cluster scaling, monitoring, etc

for additional social proof

I've been using railway since 2022 and it's been great. I host all my personal projects there and I can go from code to a url by copy-pasting my single dockerfile around.

Re: So you want to build your own data center

#36

Post by j45 »

xiconfjs wrote: post #31
jonatron wrote: post #16

Why would you call colocation "building your own data center"? You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean?

I have to second this. While it takes mich effort and in-depth knowledge do build up from an “empty” cage it’s still far from dealing with everything from building permits, to plan and realize a data center to code including redundant power lines, AC and fibre. Still kudos going this path in the cloud-centric time we live in.

Having been around and through both, setting up a cage or two is very different than the entire facility.

Re: So you want to build your own data center

#38

Post by renewiltord »

More to learn from the failures than the blog haha. It tells you what the risks are with a colocation facility. There really isn't any text on how to do this stuff. The last time I wanted to build out a rack there aren't even any instructions on how to do cable management well. It's sort of learned by apprenticeship and practice.

Re: So you want to build your own data center

#39

Post by ca508 »

sitkack wrote: post #29
ca508 wrote: post #25

Earlier quoted context omitted.

> It would be nice to have a lot more detail I'm going to save this for when I'm asked to cut the three paras on power circuit types. Re: standardising layout at the rack level; we do now! we only figured this out after site #2. It makes everything so much easier to verify. And yeah, validation is hard - manually doing it thus far; want to play around with scraping LLDP data but our switch software stack has a bug :/…

Wonderful! The edge cases are the gold btw, collect the whole set and keep them in a human and machine readable format. I'd also go through and using a color coded set of cables, insert bad cables (one at a time at first) while the system is doing an aggressive all to all workload and see how quickly you can identify faults. It is the gray failures that will bring the system down, often multiple as a single failure w…

Long running hosts at the moment, but we can drain most workloads off a specific host/rack if required and reschedule it pretty fast. We have the advantage of having a custom scheduler/orchestrator we've been working on for years, so we have a lot of control on that layer than with Kube or Nomad.

Re: Live Migration We're working on adding Live Migration support to our orchestrator atm. We aim to have it running this quarter. That'll makes things super seamless.

Re: kernels We've already seen some perf improvements somewhere between 6.0 and 6.5 (I forget the exact reason/version) - but it was some fix specific to the Sapphire Rapids cpus we had. But I wish we had more time to science on it, it's really fun playing with all the knobs and benchmarking stuff. Some of the telemetry on the new CPUs is also crazy - there's stuff like Intel PCM that can pull super fine-grained telemetry direct from the CPU/chipset https://github.com/intel/pcm. Only used it to confirm that we got NUMA affinity right so far - nothing crazy.

Re: So you want to build your own data center

#40

Post by walrus01 »

jonatron wrote: post #16

Why would you call colocation "building your own data center"? You could call it "colocation" or "renting space in a data center". What are you building? You're racking. Can you say what you mean?

> Why would you call colocation "building your own data center"?

The cynic in me says this was written by sales/marketing people targeted specifically at a whole new generation of people who've never laid hands on the bare metal or racked a piece of equipment or done low voltage cabling, fiber cabling, and "plug this into A and B power AC power" cabling.

By this, I mean people who've never done anything that isn't GCP, Azure, AWS, etc. Many terminologies related to bare metal infrastructure are misused by people who haven't been around in the industry long enough to have been required to DIY all their own infrastructure on their own bare metal.

I really don't mean any insult to people reading this who've only ever touched the software side, but if a document is describing the general concept of hot aisles and cold aisles to an audience in such a way that it assumes they don't know what those are, it's at a very introductory/beginner level of understanding the OSI layer 1 infrastructure.

Post reply on HN