Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Maybe we’ve not improved things as much as we think we have with Devops?

biven.org

Post reply on HN
Print view View original

31–40 of 43 posts

Re: Maybe we’ve not improved things as much as we think we have with Devops?

#31

Post by falcolas »

chimeracoder wrote: post #29
falcolas wrote: post #28

Earlier quoted context omitted.

My biggest problem, well two problems with Ansible: The maintainer is unusually hostile to user contributions, especially when compared to, say, Salt. A colleague of mine tried to contribute a (fully functional, tested, and documented) change to the MySQL user module to support passing in hashes for user creation - the pull request was closed with a comment like "this can go in a community module". Nevermind that the…

I haven't had any interactions with the maintainer(s), but I absolutely agree with the point on performance from SSH[0]. That said, I do like that Ansible makes no assumptions about your target server's initial state except that it is accessible over SSH (the rest can be bootstrapped from Ansible). My use case is admittedly much simpler than what Ansible allows for: I use it almost exclusively to provision a host for…

It's my belief that as our experience with containers grows, we will quickly out-grow the capabilities of a "RUN and COPY" CM paradigm. At that point, we will fall back to more traditional configuration management tools and create containers out of the results created by those CM tools.

Re: Maybe we’ve not improved things as much as we think we have with Devops?

#32

Post by rev_bird »

PaulHoule wrote: post #2

One problem is that existing devops tools are not that good. For instance it seems insane to me that packer and vagrant exist as entirely separate products. As a java programmer I find that vagrant files are way too verbose. I am sure I could write some Ruby functions to take care of that and also to fix zillions of other things that are awful about Vagrant, but then everybody writes their own functions.

The Vagrant files we've been using have been whittled down to almost nothing -- we use Chef to provision boxes, so the vagrantfile is pretty much just "install Chef, forward ports, run role xxx." At that point, it just pulls the role (and cookbooks) out of the user's local chef repo -- and getting the Vagrant Berkshelf plugin running even removes the need to deal with downloading cookbooks at all.

Re: Maybe we’ve not improved things as much as we think we have with Devops?

#33

Post by rachelbythebay »

whistlerbrk wrote: post #14

This recent trend of articles wrt to devops it making me want to post here. Not trying to be too snarky, but why can't people just use shell scripts for automation? They are easy to repair, in a language you (should) know, reproducible, and testable. They are native to your target, supremely portable, etc, etc.

Your simple script is someone else's bad day. Imagine a shell script that needs to do a dozen things in sequence. Is it just a list of 12 commands? If so, do they even have error checking, whether by "set -e" or with explicit checks? What if it does fail and it gets restarted? Are any of the steps going to cause problems if they run twice or are they smart enough to not do anything this time? How about if you have "critical sections", such that you always have to do A-B-C, and if any of them fail, you have to undo them in reverse, C-B-A? Do any of the parts need locking so that multiple instances don't run at once? Do you need timeouts on any of your calls in case something gets stuck? What cleans up the locks if you exit uncleanly? Will another instance be able to start up and continue?

You can handle all of these things in a shell script, but that's not the question. The question is: will someone who's "just writing a quick script" think that far ahead?

My experience so far suggests the answer is usually no.

It's not the language. It's the mindset.

Re: Maybe we’ve not improved things as much as we think we have with Devops?

#34

Post by xorcist »

whistlerbrk wrote: post #14

This recent trend of articles wrt to devops it making me want to post here. Not trying to be too snarky, but why can't people just use shell scripts for automation? They are easy to repair, in a language you (should) know, reproducible, and testable. They are native to your target, supremely portable, etc, etc.

It's certainly possible to do with shell scripts and by stringing lots of small utilities such as expect together. But I've been there, and after a few years and a handful of consultants came and went, it's all a big ball of mud. I know how many of my colleagues get spaces in file names, tmp races, and error handling right in their shell scripts, and they're not many.

What Puppet et al brings to the table is a bit of structure and best practice. After a two day introduction, anyone can start making changes to our infrastructure, and I'm pretty confident that the new hire in two years' time will understand it as well. And if I need to restore a previous environment, it's all in version control, and will work since it defines state as opposed to transforming it.

But the big win with a configuration management tool is not even in the tool itself. It's having a central database of _all_ configuration in every environment.

Suddenly you can start doing things like tell an application to connect to its auth service, and it'll know which one it is or even mock it if required. Your monitoring software knows every app in every environment, and can configure itself, so it's literally impossible for someone to forget to set up monitoring for a new service.

It's a huge win. Once you used to it, you never really understand how you managed without. But you really need to go all in on it to see the benefits. Automated systems integration is possible only when you store configuration in one and only one place.

Re: Maybe we’ve not improved things as much as we think we have with Devops?

#35

Post by jtreminio »

PaulHoule wrote: post #2

One problem is that existing devops tools are not that good. For instance it seems insane to me that packer and vagrant exist as entirely separate products. As a java programmer I find that vagrant files are way too verbose. I am sure I could write some Ruby functions to take care of that and also to fix zillions of other things that are awful about Vagrant, but then everybody writes their own functions.

You're not supposed to use Vagrant for the actual provisioning of a box - use the proper tools like Puppet, Chef, Ansible, Salt, etc.

And Packer is for generating the base boxes. Packer abstracts away the virtual machine software like Virtualbox, VMWare, Parallels, etc.

Re: Maybe we’ve not improved things as much as we think we have with Devops?

#37

Post by willthames »

herge wrote: post #26
_wmd wrote: post #23

Earlier quoted context omitted.

> hard to write, hard to debug (no stack traces), have weird historical artifacts Only slightly being sarcy, but this perfectly describes my experience of Salt, Ansible and Chef, but instead of a handful of 'ps' commands to figure out what's going on, you have Python/Ruby interpreters on both sides of the SSH connection, giant gumps of weird SSH command lines, a big C++ library and multitudes of TCP ports and home-gr…

Yeah, I agree with you, the tools are lacking. My favorite to date is ansible, but it's a pain to guess the syntax to the magic language that ansible has encoded inside yaml. But then that's maybe that's more of an ansible 1.1 or 1.2 issue, not sure if it's relevant anymore. I kinda wish I could just write python scripts that would use ansible as a library.

You can use ansible as a library from python: http://docs.ansible.com/developing_api.html

Re: Maybe we’ve not improved things as much as we think we have with Devops?

#38

Post by danudey »

davexunit wrote: post #3

Unfortunately, developing and deploying web applications is still a total nightmare. To build anything useful you'll need about four distinct package managers to get all of the necessary dependencies and some mish-mash of configuration management and provisioning tools to develop it, and perhaps a different set of configuration management and provisioning tools to deploy it. We can do better than this!

Hmm, let's see…

> To build anything useful you'll need about four distinct package managers to get all of the necessary dependencies

* Ubuntu packages (dpkg via apt)

* Python packages (via pip)

* Our own software packages (via our own dependency manager)

* Custom-built software packages (e.g. nginx) or non-package software installations (e.g. activemq)

> some mish-mash of configuration management and provisioning tools to develop it

We use salt for (some of) our configuration management along with just symlinks into git repositories (where possible, e.g. MySQL and others won't follow symlinks for security)

> perhaps a different set of configuration management and provisioning tools to deploy it

We have a custom-built deployment system which we use to deploy git packages (including the configuration packages that the configuration files are symlinked into) as well as our actual software.

So… yeah. You pretty much hit the nail on the head.

Re: Maybe we’ve not improved things as much as we think we have with Devops?

#39

Post by afarrell »

whistlerbrk wrote: post #14

This recent trend of articles wrt to devops it making me want to post here. Not trying to be too snarky, but why can't people just use shell scripts for automation? They are easy to repair, in a language you (should) know, reproducible, and testable. They are native to your target, supremely portable, etc, etc.

Why do I not do everything in bash?

Because the syntax for if statements and while loops is godawful. I've taught it to myself at least 30 times.

Also, there is no debugger, so I can't step through my code like I can with pub.

Also, the tooling around automated testing is worse.

Re: Maybe we’ve not improved things as much as we think we have with Devops?

#40

Post by davexunit »

danudey wrote: post #38
davexunit wrote: post #3

Unfortunately, developing and deploying web applications is still a total nightmare. To build anything useful you'll need about four distinct package managers to get all of the necessary dependencies and some mish-mash of configuration management and provisioning tools to develop it, and perhaps a different set of configuration management and provisioning tools to deploy it. We can do better than this!

Hmm, let's see… > To build anything useful you'll need about four distinct package managers to get all of the necessary dependencies * Ubuntu packages (dpkg via apt) * Python packages (via pip) * Our own software packages (via our own dependency manager) * Custom-built software packages (e.g. nginx) or non-package software installations (e.g. activemq) > some mish-mash of configuration management and provisioning too…

Thanks for the real life example.
Post reply on HN