Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don’t use environment variables for configuration

nibblestew.blogspot.com

Post reply on HN
Print view View original

211–220 of 296 posts

Re: Don’t use environment variables for configuration

#211

Post by oxplot »

I've recently started developing unix tools with config passed entirely as arguments with a special argument called `--args-file` which takes a file or stdin and reads one arg per line.

This is of course nothing new, but such a powerful pattern:

- It lets you run things without the need for a config file (when testing/running interactively)

- but on the other hand, you can still define arguments saved in a file for more permanent setups.

- `--help` ends up providing all the doc needed to configure the tool.

- It's less magical than env vars and doesn't leak into subprocesses by mistake.

One pragmatic step further is to expand env vars using the shell `${VAR}` syntax for slightly more flexibility.

Re: Don’t use environment variables for configuration

#212

Post by icedchai »

zokier wrote: post #84
sedeki wrote: post #61

I hope I don't get downvoted, but I think the OP has a point. Command line arguments will do, and perhaps even be better. Why are there so many harsh comments about this post?

In typical Linux setups, cmdline is world-readable, but environ is not. So you never should put secrets in cmdline, but they are ok to be in environ. And that is pretty much the only difference between the two.

The environment is inherited by child processes. I think that is a very important difference.

Re: Don’t use environment variables for configuration

#213

Post by chriswarbo »

merciBien wrote: post #203
ajarmst wrote: post #144

This attitude is how we ended up with Active Directory. One assumes that the author doesn't spend much time in a command-line environment. The option to set defaults for one's normal working environment is obviously of value, and environment variables are hardly exclusive of configuration files (~/.bashrc, makefiles, ...). The theory that complex syntax and statelessness are inherently good and cost-free is naïve to…

@ajarmst is that last related idea you quoted your own? That’s a powerful expression of how I see human culture: question everything, but remember to respect the ideas of the people who came before you. There might be a baby in the bath water you’re discarding.

Obligatory eponymous law: https://en.wikipedia.org/wiki/G._K._Chesterton#Chesterton's_...

Re: Don’t use environment variables for configuration

#214

Post by josephcsible »

zokier wrote: post #90
snidane wrote: post #75

Earlier quoted context omitted.

Env vars are a built-in for key-value command line args. Why introduce some custom convoluted way into your app for kv args instead of this standardised approach. I can understand why some heavily used cli app would support a custom parameter pattern, eg. for ergonomics, but for majority of apps running as deployed services and not cli apps invoked 100 times a day, I don't see a reason to not use env args. I can run…

Environ isn't actually key-value, it's only convention that applications (/libraries) parse it in the name=value form. But the underlying mechanism is similar to cmdline: array of pointers to strings. You could parse cmdline same way if you'd want.

Don't POSIX and portable C both require key=value? If so, isn't that more than just convention?

Re: Don’t use environment variables for configuration

#215

Post by earthboundkid »

kmstout wrote: post #95

I was taught many moons ago that configuration, like ogres and onions, is best considered in layers: 1. default values: What will most users in most places find most useful/least infuriating? 2. configuration files (system-wide, then user): What will most users on this system want most of the time? What will this particular user want most of the time? 3. environment variables: How should this session (i.e., a potenti…

Yeah, the article is just confused:

> Envvars have some legitimate usages (such as enabling debug logging) but they should never, ever be used for configuring core functionality of programs.

As though logging weren't core functionality!

The actual thing that is bad is grabbing an environment variable in the middle of your program. You should grab all the configuration in one place and use it to configure local state that is transparently passed around. Furthermore, flags, env vars, and config files are all just maps from strings to configuration, so you should use some system that can transparently layer them on top of one another. All of my new CLIs use flags first and fall back to ENV vars if the flag wasn't set.

Re: Don’t use environment variables for configuration

#216

Post by TrueDuality »

gnulinux wrote: post #205
kodah wrote: post #187

Earlier quoted context omitted.

On the facade environment variables may seem like they're orthogonal to global variables but they're not. Environment variables are scoped to the current process. This could be your shell, but it could also be a web server. This doesn't make them leak proof, but unlike global variables, environment variables have a scope. Environment variables are also used more widely as an API. Many CLIs have a command that when se…

I don't get this. From the same perspective you can argue global variables have scope too since they are "scoped to the current process". It's not like other processes can access your global variable, that's a very low bar for a scope.

The difference here is that a running shell (including the shell environments that services run with in) are an abstraction layer about managing processes. Within that abstraction layer, each process can have its own unique environment variables that they can change and manipulate independently. That makes them not global.

Within a process the top level abstraction is the process itself, and anything underneath (class, method, function) will be impacted if another sub-abstraction makes a change to a global variable.

Re: Don’t use environment variables for configuration

#217

Post by salawat »

merciBien wrote: post #203
ajarmst wrote: post #144

This attitude is how we ended up with Active Directory. One assumes that the author doesn't spend much time in a command-line environment. The option to set defaults for one's normal working environment is obviously of value, and environment variables are hardly exclusive of configuration files (~/.bashrc, makefiles, ...). The theory that complex syntax and statelessness are inherently good and cost-free is naïve to…

@ajarmst is that last related idea you quoted your own? That’s a powerful expression of how I see human culture: question everything, but remember to respect the ideas of the people who came before you. There might be a baby in the bath water you’re discarding.

It's a restatement of the lesson of Chesterton's Fence.

There is a fence somewhere you wish to get rid of. Nobody around knows why it was there in the first place. No one will let you tear it down until somebody figures out why it was put there in the first place.

The important take away, is that before changing something, one should understand the history of a thing, or one runs the risk of being a victim of the same problem the unknown thing was put in place to address.

Life is way more complicated than the umwelt of any one individual, so it is not safe to just change something without doing the footwork to understand what led it ro be in the first place.

Demonstrate that work has been done, and generally, no one will get in your way.

Re: Don’t use environment variables for configuration

#218

Post by hnarn »

I don't understand how a blog post like this can garner, at time of writing, a hundred upvotes. Normally these types of strong statements "Don't do X/Don't use Y" are attention seeking titles with inversely proportional interestingness of its content. That's the first red flag. The second red flag is that the article is not proof-read: from just the first few minutes of skimming, I found two errors that make the text jarring to read (Persistance->Persistence & [you] can not now -> [you] can not know).

Thirdly, the entire central point makes no sense. The author presents this argument to illustrate why environment variables are confusing:

> The environment is now different. What should the program do? Use the old configuration that had the env var set or the new one where it is not set? Error out? Try to silently merge the different options into one? Something else?

The answer is obvious to anyone that knows what environment variables are and how they work: if the variable is set, it should use it, and if it's not, it shouldn't.

The author goes on (again, spelling error) to state that:

> For comparison using JSON configuration files this entire class os (sic) problems would not exist.

What is the practical difference between using a JSON formatted text file containing settings, and a text file containing environment variables and their definitions? For the situation we're discussing here, the answer is frustratingly simple: there is none. This post is a waste of time.

Re: Don’t use environment variables for configuration

#219

Post by nikisweeting »

maccard wrote: post #73
Aeolun wrote: post #47

Earlier quoted context omitted.

> a command line build tool used by individuals that takes arguments and caches them in a per-project file That is literally one of the worst ideas I’ve ever heard of. And I’ve heard many bad ideas recently.

What's the alternative? That's how make, cmake, msbuild, ninja,docker, and basically every other build tool I'm aware of works

Docker doesn't do that. You can create a .env file, but it only reads the options you give it, Docker never writes/caches to that file.
Post reply on HN