Live data from Hacker News

How to take down production with a single Helm command

ounapuu.ee

11–18 of 18 posts

Re: How to take down production with a single Helm command

#12
post #11

> Apparently the version of Helm packaged in Fedora Linux included a patch that introduced this issue Let's translate: how to take down whatever by living on the bleeding edge.

Fedora is hardly bleeding edge

I always thought of Fedora as QA for red hat.

Re: How to take down production with a single Helm command

#13
post #5

I've seen enough of these over the last two decades, between Ubuntu/Debian and RedHat, that I use and inherently trust other distros more. Just compile the source packages. If there is some issue, file a bug. Monkey patching everything silently leads to things like this, which I've never experienced with distros like Arch, Void, or Solus.

Debian is very very diligent about patching out phone-home and expiration timers and other such “developer knows best” misfeatures. Some apps also include autoupdate (aka solarwinds style RCE) which they also remove.

Overall they do way more good than harm. Trashing them because they sometimes make mistakes is probably incorrect.

Re: How to take down production with a single Helm command

#15
post #13
post #5

I've seen enough of these over the last two decades, between Ubuntu/Debian and RedHat, that I use and inherently trust other distros more. Just compile the source packages. If there is some issue, file a bug. Monkey patching everything silently leads to things like this, which I've never experienced with distros like Arch, Void, or Solus.

Debian is very very diligent about patching out phone-home and expiration timers and other such “developer knows best” misfeatures. Some apps also include autoupdate (aka solarwinds style RCE) which they also remove. Overall they do way more good than harm. Trashing them because they sometimes make mistakes is probably incorrect.

But when they get it wrong, they can get it catastrophically wrong - like breaking ssh-keygen so that for given set of parameters there existed only 2^32 keys.

Re: How to take down production with a single Helm command

#16
That's a good argument for not running IaC tools on your own workstation but on a dedicated machine where the environment is also somehow vetted.

On the other hand I am not sure how to actually vet all used tools. For instance if you take Helm there are many pointers to repos for various Linux distros and other OSs: https://helm.sh/docs/intro/install/

Who to trust? While the Helm page says "members of the community" are in charge of the Ubuntu repo the Fedora repo is called an "official repository". Presumably judging from the OP that means "official Fedora repo" opposed to "official Helm repo".

So essentially there are no repos of the Helm maintainers which makes installing and auto-updating it on any Linux distro virtually impossible.

Re: How to take down production with a single Helm command

#17
post #11

> Apparently the version of Helm packaged in Fedora Linux included a patch that introduced this issue Let's translate: how to take down whatever by living on the bleeding edge.

Fedora is hardly bleeding edge

Apparently bleeding enough in this case.

Re: How to take down production with a single Helm command

#18

> Apparently the version of Helm packaged in Fedora Linux included a patch that introduced this issue Let's translate: how to take down whatever by living on the bleeding edge.

And increase the bloodshed by, apparently, configuring your staging environment significantly different than your production environment. An instigating the bloodshed by doing something first in production and then testing it in staging (sure, the first time staging wouldn't have caught it, but maybe then you could look to see where you misconfigured staging...)
Post reply on HN