Live data from Hacker News

The golden age of configuration languages

cosminilie.ro

51–53 of 53 posts

Re: The golden age of configuration languages

#51
post #40

Earlier quoted context omitted.

As an old school sysadmin for a long time, the feeling I've always gotten from DevOps is that it is like anything else in the big ball of mud we call "computer science": a new generation re-inventing the wheel. To be sure, the scale is larger than ever before. But history is replete with "IT in the large". Just look at mainframes, quietly ticking away for decades on loads that people are re-discovering how to handle.…

That's not the way I interpret 're-inventing the wheel.' I don't see it as the next gen thinking they are smarter, but rather questioning the assumptions built into the last gen. Mainframes vs Distributed (cloud, kubernetes, etc) systems have different behaviors at different scales and are useful for different kinds of applications and business sizes. A perfect example is the startup economy. I can spool up a service…

So yes, the spread of powerful computing certainly is to be lauded. I'm not denying that, but I feel it's more to do with a driving down of costs than anything. If (and I admit that's a big if) big iron vendors had paid attention and not rested on their laurels, I truly believe they could have easily morphed into "cloud" providers, and possibly even brought it on sooner, like a decade earlier.

But the activity of keeping secrets is still with us, even in this age of widespread FLOSS. Just look up how many software patents are still held. It's no good trumpeting your organization's advantages if everyone has the same advantages.

Re: The golden age of configuration languages

#52

Earlier quoted context omitted.

Automating repeatable things to remove the need for some institutional knowledge is a good thing.

This I will give to DevOps - it's hyper focus on automation, change management and repeatability are all good things. As someone with a leg in each domain (software developer by day, system administrator by night), I've really felt that both areas could learn a lot from each other. These days with version control being as quick and easy as it is, there's no reason not to use it for everything. Configuration as code i…

It's gone too far though. Where I work everything is automated, even the setup of things where we only have one of them. It's all a giant pile of Ansible scripts, some of which are sort of half maintained.

This kind of fanatically DevOps-y approach has a few problems:

1. It's slow and expensive to develop all these scripts.

2. It encourages "disposable infrastructure" in which people think, gee, I need a test version of {very big complicated thing}, so I'll just run these scripts and create an absurdly expensive duplicate that they then forget to shut down. You end up with a cloud filled with VMs that were used once, by one guy, for twenty minutes for a demo, and then never properly deleted.

3. There's no real discoverability or way to iteratively figure out what you want with these things. If you aren't entirely sure how to set something up, you can't really do it by coding it because you'd end up being limited by the tools. Whereas shells and vim and such are designed for interactive work. But then once you've got it set up, for many tasks you hopefully don't need to set it up again anytime soon, so at that point automation is just pointless overhead. It'd be quicker to just drop some notes in a wiki.

With respect to the article, it did very much remind me of the great wheel turning. I used to be an SRE at Google and I saw there two relevant trends.

One was that when I joined they still had some parts of the infrastructure managed by a giant Python script called the Babysitter. The Babysitter was in theory split into configuration and implementation aspects, but somehow not really. It's dead for perhaps 12 or 13 years now so many Googlers would never have encountered it. The replacement system (Borg) used an intentionally limited configuration language, partly due to the experience of the Babysitter in which people had full programmability, and used it! But the borgcfg language rapidly started sprouting various kinds of programmability features too, and in fact one day someone demonstrated you could build Conway's Game of Life in it. Seems like this new call to use full programming languages to create configuration is another turn of this wheel.

The other was hitting the limits of what was automatable. One of the tasks I had to do in my job was to set up some services in new clusters, and shut them down in old ones. This was a multi-step process that was very time consuming and annoying, so there were many attempts to automate it. Unfortunately most were unconvincing, partly because the speed at which new clusters needed to be set up was somehow sort of aligned with the speed at which the general set of tasks was itself changing. So you could attempt to automate the task, but by the time you'd finished the steps involved were already changing, so the next time you wanted to run your script it was wrong. Also: setting up clusters of hundreds of machines isn't something that's easy to test virtually (there were no "mock datacenter" libraries back then).

In the end, what I noticed was the big automation wins came from the infrastructure teams when they improved the underlying systems to be more robust. Us glorified sysadmins wrote a lot of scripts, but it was often hard to prove they really saved a lot of time when the cost of implementing and changing them was taken into account.

Re: The golden age of configuration languages

#53
post #52

Earlier quoted context omitted.

This I will give to DevOps - it's hyper focus on automation, change management and repeatability are all good things. As someone with a leg in each domain (software developer by day, system administrator by night), I've really felt that both areas could learn a lot from each other. These days with version control being as quick and easy as it is, there's no reason not to use it for everything. Configuration as code i…

It's gone too far though. Where I work everything is automated, even the setup of things where we only have one of them. It's all a giant pile of Ansible scripts, some of which are sort of half maintained. This kind of fanatically DevOps-y approach has a few problems: 1. It's slow and expensive to develop all these scripts. 2. It encourages "disposable infrastructure" in which people think, gee, I need a test version…

On your points 1 and 2, absolutely I recognize those problems. The second appears to be a user education or non-bounded task problem (can't you just have the cloud shutdown VMs that aren't accessed for a specified time period?). I do like the concept of disposable infrastructure, but it is a tool like seven-league boots that one must think very carefully while using.

Addressing the first point, well, that's almost getting into business level ROI calculations, which at least on an individual developer level might be reasonably approximated by things like "is it worth the time?"[0], but that's never been a perfectly answerable easy question; for one thing, you might need to consider that the time spent is learning that will pay off in the future. I like to think of it as "technical investing" as opposed to technical debt.

As for three, that's a big problem, no matter the scale or project. I do not have the benefit of your experience with software in the large (thank you for your stories!), but even on a smaller project I'm working on recently I run into this problem: we have git repos. We have documentation in the repos. But to get to that documentation, you have to know how to get to the repos, and where do you document that? I tried imagining automating things completely, and the answer I came to was ultimately I'd have to build an AI to on-board new developers to the project. Not ideal.

And like you said, the scaffolding surrounding projects is frustrating too. Again, same project, but this time I'm digging into the build system. Turns out it was converted to make from the previous build system that appeared to be mostly shell scripts, yet the shell scripts are still around, unused, but like little land mines, waiting for someone to tinker with them and waste a bunch of time.

[0] - https://imgs.xkcd.com/comics/is_it_worth_the_time.png

Post reply on HN