If you want to know more about some of the technical details, we wrote something up: https://www.systeminit.com/blog-five-breakthroughs
System Initiative: Second Wave DevOps
11–20 of 80 posts
Re: System Initiative: Second Wave DevOps
#12>Doing “DevOps work” is unquestionably the worst part of building a modern application. It’s full of tiny papercuts, indignities we suffer in our toolchains, our feedback loops, and our software. It’s a city of brutalist buildings filled with sharp-edged couches pretending to be comfortable. Think of all the advances in how we interact with tools in other domains - then take a look at the way you build, deploy, and o…
> I dunno, usually I find databases and migrations to be the hard part.
For me, the following tools make that a joy: - Postgres as the database, which is very predictable and extremely reliable; - Migrations with Ruby on Rails, that have just the right balance between a convenient DSL and letting you write SQL when necessary; - The strong_migrations gem that catches in development unsafe commands to run in production, and explains how to make them safe
Re: System Initiative: Second Wave DevOps
#13After reading this post, I've browsed the site. I'm not sure how this is anything but significantly worse than the current model? I've been around long enough to know that any "no code" style interface or GUI are typically the _problem_ not the solution. Regardless of the code they export, you end up with fat fingers, misclicks, forgotten UI paths to follow... Taking a software eng approach to shipping infra is a sta…
> Devops doesn't have to be hard. Infrastructure doesn't have to be complex
That's simply not true for anything larger than a few services and a small dev team. The cloud is very complex to do right when you focus on security, performance, and scalability. And Terraform invariably devolves into a nightmare when you have a ton of resources with dependencies between them.
Re: System Initiative: Second Wave DevOps
#14>Doing “DevOps work” is unquestionably the worst part of building a modern application. It’s full of tiny papercuts, indignities we suffer in our toolchains, our feedback loops, and our software. It’s a city of brutalist buildings filled with sharp-edged couches pretending to be comfortable. Think of all the advances in how we interact with tools in other domains - then take a look at the way you build, deploy, and o…
Fundamentally, I think the “why” is still the fact that DevEx started with devs, and DevOps started as a collaboration with Ops people, who already were not speaking the same language of robustness that we do. Which is a little weird, and probably part of the friction between the groups. They historically took on the reliability role, if nobody else did, but they were implementing reliability on top of a house of car…
"They historically took on the reliability role, of nobody else did, but they were implementing reliability on top of a house of cards, which is a kind of hypocrisy that makes even mediocre devs bristle. Don’t lecture me on robust software, boyo. Your tools are made of string cheese and staples."
I don't know why you'd blame ops for the crappyness of the tools they have at their disposal.
Yes, Ansible, Salt, Puppet, and Chef are spaghetti-code inducing congealed messes of design. So are large collections of complex shell scripts.
So what's the alternative? What spherical cow of a configuration management tool from Platonic dev heaven shall be foisted on us this time?
I'm sure it'll be super clean and elegant this time, unlike the last thousand shitty tools they made.
And don't get me started on devs that think they're qualified to do ops when all they know is their language of choice (if even that) and have never thought about the network, security, capacity, redundancy, failover, reliabililty, hardware, backups, the rest of the company or other users.
Re: System Initiative: Second Wave DevOps
#15After reading this post, I've browsed the site. I'm not sure how this is anything but significantly worse than the current model? I've been around long enough to know that any "no code" style interface or GUI are typically the _problem_ not the solution. Regardless of the code they export, you end up with fat fingers, misclicks, forgotten UI paths to follow... Taking a software eng approach to shipping infra is a sta…
Text-based interfaces still have the same issue with being able to make a typo or tab completing without checking. Seems like the major advantage is that these text based tools are able to be versioned well through scripts. This might be fixed with a GUI version of autohotkey, turning these gui interactions into a script.
Re: System Initiative: Second Wave DevOps
#16The core value proposition is really valuable and the demo video nailed it. Change propagation as requirements change is ripe for error.
Have been following progress, excited for where it goes.
Re: System Initiative: Second Wave DevOps
#17After reading this post, I've browsed the site. I'm not sure how this is anything but significantly worse than the current model? I've been around long enough to know that any "no code" style interface or GUI are typically the _problem_ not the solution. Regardless of the code they export, you end up with fat fingers, misclicks, forgotten UI paths to follow... Taking a software eng approach to shipping infra is a sta…
Text-based interfaces still have the same issue with being able to make a typo or tab completing without checking. Seems like the major advantage is that these text based tools are able to be versioned well through scripts. This might be fixed with a GUI version of autohotkey, turning these gui interactions into a script.
Mandated peer review, planned actions, and automated risk evaluation are part of our infra pipeline. This typically doesn't exist outside of software dev style pipeline.
Re: System Initiative: Second Wave DevOps
#18After reading this post, I've browsed the site. I'm not sure how this is anything but significantly worse than the current model? I've been around long enough to know that any "no code" style interface or GUI are typically the _problem_ not the solution. Regardless of the code they export, you end up with fat fingers, misclicks, forgotten UI paths to follow... Taking a software eng approach to shipping infra is a sta…
"Deploying every day isn't _that_ difficult"
Just don't ever ask to roll back...
Re: System Initiative: Second Wave DevOps
#19After reading this post, I've browsed the site. I'm not sure how this is anything but significantly worse than the current model? I've been around long enough to know that any "no code" style interface or GUI are typically the _problem_ not the solution. Regardless of the code they export, you end up with fat fingers, misclicks, forgotten UI paths to follow... Taking a software eng approach to shipping infra is a sta…
> Devops doesn't have to be hard. Infrastructure doesn't have to be complex That's simply not true for anything larger than a few services and a small dev team. The cloud is very complex to do right when you focus on security, performance, and scalability. And Terraform invariably devolves into a nightmare when you have a ton of resources with dependencies between them.
I'm definitely not google scale, but we're global, in over 300 cities spanning ~30 countries. On an avg day we process well over 25k rps on multiple services. Simple architecture and IaC like terraform is exactly how we manage the dependencies. It's the solution, not the problem.
Re: System Initiative: Second Wave DevOps
#20After reading this post, I've browsed the site. I'm not sure how this is anything but significantly worse than the current model? I've been around long enough to know that any "no code" style interface or GUI are typically the _problem_ not the solution. Regardless of the code they export, you end up with fat fingers, misclicks, forgotten UI paths to follow... Taking a software eng approach to shipping infra is a sta…
> Devops doesn't have to be hard. Infrastructure doesn't have to be complex That's simply not true for anything larger than a few services and a small dev team. The cloud is very complex to do right when you focus on security, performance, and scalability. And Terraform invariably devolves into a nightmare when you have a ton of resources with dependencies between them.
Esp. as you start splitting up your statefiles!
However, I do think that this is mostly essential complexity, rather than accidental one. We're now building systems that are way more secure and/or scalable than before. Least possible network access and permissions everywhere already add a bunch of complexity. Pushing complexity from our code to managed cloud offerings does its part, too. But all of this can be tamed very well with modules and reusable components.
That said, if you're scaling Terraform, I do recommend you to check out the tools that have sprung up in the recent years to manage it. I'll personally recommend Spacelift[0] (see disclaimer). It can help you orchestrate your statefiles once you start having many of them (even tens or hundreds of statefiles in a single workflow are no problem) using stack dependencies, help team members self-serve through blueprints, automate all the things through OPA policies, and generally help you scale your Terraform usage to a larger team.
[0]: https://spacelift.io
Disclaimer: Software Engineering Team Lead at Spacelift, so take the recommendation with a fair grain of salt; I do legitimately think it's a great product though. If you'd like to reach out, feel free to do so through the website or the contact details in my profile.