Don’t use environment variables for configuration • Hacker News Board

Re: Don’t use environment variables for configuration

#241

Post by axaxs » Thu, Apr 01, 2021, 7:24 PM UTC

rocqua wrote: post #137

silisili wrote: post #13

This post seems to ramble without much substance. The best argument, perhaps only valid argument, is lack of an array type. Easy to work around. The rest seems misguided or ridiculous. >There is no way to know which one of these is the correct form What? Of course there is. The author is also calling it mutable global state, and seems to reference an application becoming confused when ENV isn't set. This reads to me,…

To be fair, environment variables are mutable global state across a single shell session.

Howso? A running process won't affect your current env, and changing your current env doesn't affect the running process.

Re: Don’t use environment variables for configuration

#242

Post by benlivengood » Thu, Apr 01, 2021, 7:30 PM UTC

Configuration should be parseable. Unix philosophy has been to use configuration files for anything much more complex than flags, which seems reasonable to me. Don't try to force structured configuration into an arrays of strings.

Traditionally, I see flags as being very application-specific and environment variables being very generic. Environment variables control the behavior of shared libraries and the interaction with the operating system. That separation frees applications from worrying about name collisions between application and library configuration.

The tradeoff Unix made between unstructured configuration (e.g. main(void *arg)) and fully structured configuration was probably wise. Arrays of strings with quoting rules are pretty flexible and human readable. It doesn't gives the full flexibility of the extreme approaches but has aged well.

For example, anyone who is annoyed by environment variables is free to wrap every program they care about in a shell script that sets the environment from the flags they want. Anyone who hates flags can pass environment variables to their version of the script with a bunch of --flag=${MY_PRECIOUS} inside.

Re: Don’t use environment variables for configuration

#243

Post by mixmastamyk » Thu, Apr 01, 2021, 8:02 PM UTC

CGamesPlay wrote: post #128

zajio1am wrote: post #106

Strongly disagree. Environment variables are, IMHO, best tool for some simple configuration in unix. They match perfectly with behavior of the ecosystem and other tools in it (like unix shell). Yes, if your OS is some unversal JS machine, then JSON would be better, if it is Lisp machine, then you would use S-expressions, but on Unix machine, environment/args are way to go. There are two realistic alternatives - confi…

> Perhaps the ideal tool would allow every option to be set/changed from config file, environment and argument. This is exactly what the most widely used golang configuration library does: https://github.com/spf13/viper

I made something similar for Python, with an animal theme too, ha:

https://pypi.org/project/tconf/

Re: Don’t use environment variables for configuration

#244

Post by tomphoolery » Thu, Apr 01, 2021, 8:13 PM UTC

Environment variables, when used correctly, tend to work just about everywhere. That's why they're in use. It's the same reason why programs still install to `/usr/bin`, even though we're all very much aware that `/usr` is not "user direectories" and the whole idea of "userspace" vs "kernel space" is irrelevant in 99% of *nix installs these days (including containers).

Re: Don’t use environment variables for configuration

#245

Post by Lvl999Noob » Thu, Apr 01, 2021, 8:16 PM UTC

OJFord wrote: post #176

yrro wrote: post #131

Earlier quoted context omitted.

Why I don't like environment variables: 1. I worry about programs dumping all their environment variables to log files - credentials are now on disk, ingested into log storage... 2. Environment variables are inherited by child processes by default. This is undoubtable useful. But it can also cause problems. I wish the ghosts of unix past had forseen the need for a way to mark particular variables as, say, 'sensitive'…

I used to agree with (2), but now I think Meh, it's an implementation detail whether the program uses my environment variable 'directly' or with a child process, it's not meaningful to make that distinction. When it is meaningful (and this is supported today) is to set them just for specific programs/invocations, rather than exporting for a long-running interactive shell (and everything within it) willy-nilly. More i…

Iiuc you are advocating for setting env vars at the call site, like `FOO_SECRET=hunter2 foobar`? In that case, why not just use command-line args and call it like `foobar --secret=hunter2`?

Re: Don’t use environment variables for configuration

#246

Post by jschwartzi » Thu, Apr 01, 2021, 8:19 PM UTC

npsimons wrote: post #158

yrro wrote: post #131

Earlier quoted context omitted.

Why I don't like environment variables: 1. I worry about programs dumping all their environment variables to log files - credentials are now on disk, ingested into log storage... 2. Environment variables are inherited by child processes by default. This is undoubtable useful. But it can also cause problems. I wish the ghosts of unix past had forseen the need for a way to mark particular variables as, say, 'sensitive'…

My first thought on the headline are specialized concerns of the above: environment variables are an attack surface. If you use them for configuration, it's all too easy for an attacker to modify them without the victim knowing. Just look at issues with LD_PRELOAD: https://attack.mitre.org/techniques/T1574/006/ That said, I agree with GP that environment variables are super useful and super simple. But I've also been…

There's actually a long list of variables that are unset when invoking sudo to prevent these kinds of attacks. Systemd will also start programs with a very minimal environment that isn't inherited from any shells. You then have to specify environment variables explicitly as part of the unit file. You can also specify environment variables in environment files.

Re: Don’t use environment variables for configuration

#247

Post by fatbird » Thu, Apr 01, 2021, 8:20 PM UTC

His example:

  $ SOME_ENVVAR=... some_command

Is wrong. SOME_ENVVAR is set only for the invocation some_command; at the next prompt, the value of SOME_ENVVAR is what it was before the invocation. To persist the setting of the value you need to export it:

  $ export SOME_ENVVAR=... some_command

Re: Don’t use environment variables for configuration

#248

Post by michaelmior » Thu, Apr 01, 2021, 8:21 PM UTC

kodah wrote: post #187

ajarmst wrote: post #144

This attitude is how we ended up with Active Directory. One assumes that the author doesn't spend much time in a command-line environment. The option to set defaults for one's normal working environment is obviously of value, and environment variables are hardly exclusive of configuration files (~/.bashrc, makefiles, ...). The theory that complex syntax and statelessness are inherently good and cost-free is naïve to…

On the facade environment variables may seem like they're orthogonal to global variables but they're not. Environment variables are scoped to the current process. This could be your shell, but it could also be a web server. This doesn't make them leak proof, but unlike global variables, environment variables have a scope. Environment variables are also used more widely as an API. Many CLIs have a command that when se…

> Environment variables are scoped to the current process.

Obviously this is true in a sense. But for practical purposes, it depends on how the environment variables are set. For example, if I set them in my ~/.bash_profile, they're scoped to all bash processes that my current user runs. If I put them in /etc/profile, they're effectively global.

Re: Don’t use environment variables for configuration

#249

Post by jschwartzi » Thu, Apr 01, 2021, 8:22 PM UTC

OJFord wrote: post #176

yrro wrote: post #131

Earlier quoted context omitted.

Why I don't like environment variables: 1. I worry about programs dumping all their environment variables to log files - credentials are now on disk, ingested into log storage... 2. Environment variables are inherited by child processes by default. This is undoubtable useful. But it can also cause problems. I wish the ghosts of unix past had forseen the need for a way to mark particular variables as, say, 'sensitive'…

I used to agree with (2), but now I think Meh, it's an implementation detail whether the program uses my environment variable 'directly' or with a child process, it's not meaningful to make that distinction. When it is meaningful (and this is supported today) is to set them just for specific programs/invocations, rather than exporting for a long-running interactive shell (and everything within it) willy-nilly. More i…

This is pretty much how systemd works. You can specify secrets that are retrieved from somewhere else and provided to the process in the environment it is started with. So you could do exactly this with the right unit configurations.

Re: Don’t use environment variables for configuration

#250

Post by alerighi » Thu, Apr 01, 2021, 8:22 PM UTC

sedeki wrote: post #61

I hope I don't get downvoted, but I think the OP has a point. Command line arguments will do, and perhaps even be better. Why are there so many harsh comments about this post?

Command line parameters should be used for things that change frequently, not for configuration variables. You don't want to pass 20 command line parameters every time you invoke a program. You rather want to set your environment variables one time in your shell initialization script (.bashrc, .zshrc, whatever).

You can say there are the configuration files, and sure a complex program should have its configuration file. The problem of configuration files is that they all use a different syntax that you have to learn, you have to write them, back them up, copy around, etc.

Also you don't have a simple way of overriding a configuration file parameter for one invocation of the program, that isn't modifying the configuration file. With environment variables is simple: set them before executing the program.

Last thing is that environment variables aren't specific to a particular program: they are readable by every program executed in that shell. A configuration file cannot be easily shared, since one program can change its format to make it no longer backwards compatible, so you can't rely on reading other program configuration files. While you can with environment variables.