Don’t use environment variables for configuration • Hacker News Board

Re: Don’t use environment variables for configuration

#191

Post by yrro » Thu, Apr 01, 2021, 4:20 PM UTC

bopbeepboop wrote: post #182

yrro wrote: post #166

Earlier quoted context omitted.

Good point about command line arguments being often automatically logged! So they bad for both reasons :) Now, if you're running in k8s then you can improve your setup by mounting your secret into your container, and have your code read the credentials from the file within the mount. This just looks like another kind of config file to me :)

> And for the same reason, a child process that does the same won't inherit them in order to expose them. Wait, this is the crux of why you think it’s more secure — but actually I see the reverse problem: Dropping environmental variables is standard security practice, but dropping file access permissions is not. Most child processes read from the same set of files as the parents. How would having files rather than EN…

I'm concerned with 'I have credentials in my environment block and just dumped the whole thing to a log file'. Avoiding storing the credentials in the environment block obviously avoids that.

To be fair, unsetting sensitive environment variables after consuming them would probably also avoid that eventuality. I can count on the fingers of no hands the number of times I've seen developers do that! :)

Some other part of my process (or a child process I might launch) deliberately hunting for credentials in order to leak them is a different problem with other solutions.

In between these two cases we have mistakes like "dump config object (containing credentials) to a log file". That, too, can happen and should be avoided, what more can I say?

Re: Don’t use environment variables for configuration

#192

Post by yrro » Thu, Apr 01, 2021, 4:24 PM UTC

PeterisP wrote: post #177

yrro wrote: post #166

Earlier quoted context omitted.

Good point about command line arguments being often automatically logged! So they bad for both reasons :) Now, if you're running in k8s then you can improve your setup by mounting your secret into your container, and have your code read the credentials from the file within the mount. This just looks like another kind of config file to me :)

Embedding secrets (which should be changeable and with limited access) into container images (which should be reproducible and perhaps stored in accessible locations) sounds like not a goood idea; IMHO you definitely need the capability to have the same container use different credentials so that, for example, you can run the same container in a development or testing environment as in production, but with different…

I'm talking about doing this: https://kubernetes.io/docs/concepts/configuration/secret/#us...

Re: Don’t use environment variables for configuration

#193

Post by peterthehacker » Thu, Apr 01, 2021, 4:28 PM UTC

Pydantic has a nice way to parse/validate a collection of environment variables into native python types using type hints [0]. It solves for some of the pain points the author mentioned in a clean way.

[0] https://pydantic-docs.helpmanual.io/usage/settings/

Re: Don’t use environment variables for configuration

#194

Post by teddyh » Thu, Apr 01, 2021, 4:33 PM UTC

kmstout wrote: post #95

I was taught many moons ago that configuration, like ogres and onions, is best considered in layers: 1. default values: What will most users in most places find most useful/least infuriating? 2. configuration files (system-wide, then user): What will most users on this system want most of the time? What will this particular user want most of the time? 3. environment variables: How should this session (i.e., a potenti…

This echos nicely the traditional wisdom, also described here: http://www.catb.org/~esr/writings/taoup/html/ch10s02.html

Re: Don’t use environment variables for configuration

#195

Post by peterthehacker » Thu, Apr 01, 2021, 4:33 PM UTC

int first_argument; int second_argument;

void add_numbers(void) { return first_argument + second_argument; }

// to call it you'd do first_argument = 1; second_argument = 2; int three = add_numbers();

> This is, I trust you all agree, terrible.

This isn’t really a case against environment variables per say but against global config variables generally. Whether it comes from the environment or a config file that state will be global. So the example of adding globally defined integers is obviously silly, but if you change those variables to a username and password strings then change the function to db_connect it makes sense.

Re: Don’t use environment variables for configuration

#196

Post by Blikkentrekker » Thu, Apr 01, 2021, 4:36 PM UTC

No, they aren't mutable global state, they are paramatrers which is something many languages support: the key difference by design is that their configuration only lasts for the dynamic extent of the call.

An environment variable is not mutable global state, for if a process change one, then that change is inherited by it's children, but not by it's parent or siblings, and the change effectively stops existing once the process ends. — that's a very important design change that removes all of the problem with mutable global state.

In that sense, they are more like thread local variables that are inherited by child threads.

A configuration file is much more akin to global mutable state of a system.

Re: Don’t use environment variables for configuration

#197

Post by amelius » Thu, Apr 01, 2021, 4:37 PM UTC

Another nice one is chdir() and how it behaves between threads.

Re: Don’t use environment variables for configuration

#198

Post by marcosdumay » Thu, Apr 01, 2021, 4:39 PM UTC

marcrosoft wrote: post #157

mrkeen wrote: post #15

Easy fix (if it's the shared, mutable state which bugs you): * Create one class responsible for ingesting env vars at startup. * Call it from main, and abort early with nice messages if it fails to read something. * Now you have a nice (preferably immutable) class which guarantees the config is in a 'good state', and is self-documenting because it lists all the keys it uses to lookup env vars with.

The mutable state can be helpful. It is sometimes helpful to be able to change an app’s config without having to restart it. Ingesting the envs on startup into a class removes this ability.

Please, never do that on a server.

It's ok for interactive applications, but if you are writing a CLI command (what is different from an interactive CLI application), a system library or a deamon, don't ever let the same application that uses a configuration also change it.

When your non-interactive programs do that and anything at all goes wrong, it's basically impossible to determine the source of the problem. Also, it is common that bugs that one could just avoid triggering by configuration now become unavoidable.

(But if you mean reload the config after getting a SIGHUP or something like that, yeah, this is ok, and the best way to do that is by restarting everything on your program, even if you keep the same process, so your read-once class won't be a problem.)

Re: Don’t use environment variables for configuration

#199

Post by exabrial » Thu, Apr 01, 2021, 4:40 PM UTC

That's literally what they're for. Take for instance:

DATABASE_URL is a fail-safe method of configuring. When you install the app in dev, the app can _only_ point to the dev database. If the variable is absent, the application fails to start. If the prod URL is accidentally put in the the dev environment, firewall rules would prevent the prod connection.

Configuration is _best_ put into the environment, not the application itself.

Re: Don’t use environment variables for configuration

#200

Post by josephcsible » Thu, Apr 01, 2021, 4:40 PM UTC

"Environment variables is exactly this: mutable global state." No it isn't. Every time you start a process, it gets a set of environment variables of its own, which won't be changed by any further changes in the parent process. This is the opposite of how global variables work and is exactly how function arguments work.

The rest of the article isn't very good either. The examples of running a program with two different states and of trying to do nested escaping would both apply to any means of passing configuration.

I also find it hilarious that the author suggests using JSON configuration files instead, which actually have many of the problems that this article falsely claims that environment variables have.