Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don’t use environment variables for configuration

nibblestew.blogspot.com

Post reply on HN
Print view View original

91–100 of 296 posts

Re: Don’t use environment variables for configuration

#91

Post by emodendroket »

q3k wrote: post #88
bayindirh wrote: post #76

I have a pet peeve about this attitude. These methods are being used for decades and well understood with all their advantages and disadvantages. One day, someone comes and tells that it's bad and considered harmful , and happily tells the only right way to do it . A flame war ensues then. I'm all for moving things forward and evolution, but can't we take a milder stance and move forward in a more peaceful way? Attac…

> That thing wouldn't be a de-facto standard if it was too bad, right? I disagree. Very often, the 'easiest' option wins, not the one that necessarily is the 'best', especially in the long term. Environment variables for configurations are fast and easy, and work well on the happy path - but break down quite fast when dealing with more complex cases (eg. more complex types than strings). They have a treacherous way o…

Sure, but everyone understands that option and knows the pitfalls. This is kind of the core of the “worse is better” mindset, which somehow seems to have disappeared from the collective consciousness even though Unix is bigger than ever.

Re: Don’t use environment variables for configuration

#93

Post by haolez »

maccard wrote: post #73
Aeolun wrote: post #47

Earlier quoted context omitted.

> a command line build tool used by individuals that takes arguments and caches them in a per-project file That is literally one of the worst ideas I’ve ever heard of. And I’ve heard many bad ideas recently.

What's the alternative? That's how make, cmake, msbuild, ninja,docker, and basically every other build tool I'm aware of works

Make doesn't work like that.

Re: Don’t use environment variables for configuration

#94

Post by bayindirh »

q3k wrote: post #88
bayindirh wrote: post #76

I have a pet peeve about this attitude. These methods are being used for decades and well understood with all their advantages and disadvantages. One day, someone comes and tells that it's bad and considered harmful , and happily tells the only right way to do it . A flame war ensues then. I'm all for moving things forward and evolution, but can't we take a milder stance and move forward in a more peaceful way? Attac…

> That thing wouldn't be a de-facto standard if it was too bad, right? I disagree. Very often, the 'easiest' option wins, not the one that necessarily is the 'best', especially in the long term. Environment variables for configurations are fast and easy, and work well on the happy path - but break down quite fast when dealing with more complex cases (eg. more complex types than strings). They have a treacherous way o…

> I disagree. Very often, the 'easiest' option wins, not the one that necessarily is the 'best', especially in the long term.

Thank you for your disagreement and discussion, honestly. Actually, I think using environment variables are a burden. Needs more documentation, more explicit warnings, a lot of handling, etc.

So, environment variables are not the easiest way out there. Especially when almost any programming language has nice config file libraries out of the box. Instead these variables are added as a convenience feature for some frequent scenarios where tool needs to adapt itself to the environment it needs to run in, just before starting or needs to be run repeatedly with small, transient changes to the config.

> Furthermore, as long as environments have existed in UNIX and UNIX-like derivatives, their usage to configure the bulk of the behaviour of a service/program are relatively new.

This is not what I see in my career. Bulk of the applications we installed and ran used some forms of environment variables for runtime configuration of the tool/application.

The reason for that the variable had a great deal of effect in the behavior of the program (which was generally scientific) and making multiple runs without modifying a file very effective. You need these runs to conduct research BTW, and you're on a cluster and jobs run long and whatnot.

TBH, most of these applications also had configuration files or "sensible defaults" and they either created their default files if there was none. And if there was a file, the environment variable was acting as an override.

So I had experimental software, fixed most of the parameters in the file and tried some other things by overriding some parameter(s) with an environment variable. Nothing was abused or misused.

> Indeed, sometimes it seems like 90% of the Docker code out there is converting environment variables into configuration files.

I've never seen it TBH, and if that's not documented well, it would be a big bag of fun for the users of that code.

Re: Don’t use environment variables for configuration

#95

Post by kmstout »

I was taught many moons ago that configuration, like ogres and onions, is best considered in layers:

1. default values: What will most users in most places find most useful/least infuriating?

2. configuration files (system-wide, then user): What will most users on this system want most of the time? What will this particular user want most of the time?

3. environment variables: How should this session (i.e., a potentially large series of related executions) be tailored?

4. command line options: What is most useful for this particular run?

I was also taught that:

- figuring out how to go from an option to the name of a corresponding environment variable to a line in a config file should be both straightforward and well documented; and

- sometimes you need a more complex configuration than is cleanly supportable through any other method than a file. In such a case, the location of that file can itself be passed through options and the environment.

Re: Don’t use environment variables for configuration

#96

Post by emodendroket »

jillesvangurp wrote: post #60
trey-jones wrote: post #55

This flies in the face of pretty much every opinion I've heard from experienced developers in the past 5 years. Once someone said, "You should be using ENV for configuration" I started doing it, and I found it to be a better solution than I previously had. I am also open to dedicated config files, whether they set ENV vars or not. I'm open to the idea that ENV is not the only way, and I certainly believe that there a…

Yes. If you package software up for deployment on some server (i.e. you use something like Docker), environment variables are the easiest/only configuration mechanism at your disposal. Packages that don't support this, need some workarounds (e.g. dockerize to template some config file using environment variables) to be packaged up; which is annoying and extra work. Decent server software comes prepackaged in docker f…

Docker certainly supports mounting a configuration file on the host system. Might be a little messier but it’s not impossible.

Re: Don’t use environment variables for configuration

#97

Post by q3k »

jillesvangurp wrote: post #60
trey-jones wrote: post #55

This flies in the face of pretty much every opinion I've heard from experienced developers in the past 5 years. Once someone said, "You should be using ENV for configuration" I started doing it, and I found it to be a better solution than I previously had. I am also open to dedicated config files, whether they set ENV vars or not. I'm open to the idea that ENV is not the only way, and I certainly believe that there a…

Yes. If you package software up for deployment on some server (i.e. you use something like Docker), environment variables are the easiest/only configuration mechanism at your disposal. Packages that don't support this, need some workarounds (e.g. dockerize to template some config file using environment variables) to be packaged up; which is annoying and extra work. Decent server software comes prepackaged in docker f…

> Dedicated configuration files only make sense if you assume a writable file system is there.

I disagree. It's pretty typical in containerized environments (in my experience) to pass in (eg. mount) a config generated by whatever configuration management system into a container for it to load its configuration from.

This has the following advantages over env var configs:

- support for more complex expressions than string -> string maps (eg., configuring an IP blocklist)

- less chance of mistakes stemming from typos (eg., 'FOO_LODGIR' instead of a 'FOO_LOGDIR' in an environment variable will likely be silently ignored by a service, while a 'lodgir' key in a config file will cause an error in most serious config parsers that I've seen)

- working against a schema - if you use something like openapi, json-schema or protobuf/prototext to define your config format, you can use this schema to check/generate the config from other code, and even use it as an automatic source of documentation for the configuration format

- hot reloads of configs - once started with env vars, the env vars cannot be (easily) changed, while files can easily be changed (on mutable FS, or from an external source like ConfigMaps/Secrets in k8s), watched and reloaded from, or even used as a signal that the software should restart

Re: Don’t use environment variables for configuration

#99

Post by zabzonk »

bayindirh wrote: post #76

I have a pet peeve about this attitude. These methods are being used for decades and well understood with all their advantages and disadvantages. One day, someone comes and tells that it's bad and considered harmful , and happily tells the only right way to do it . A flame war ensues then. I'm all for moving things forward and evolution, but can't we take a milder stance and move forward in a more peaceful way? Attac…

> These methods are being used for decades and well understood with all their advantages and disadvantages.

Just because something has been used for decades does not make it good - for example, avoidable mutable state. And it certainly does not make it well-understood - as a consultant the number of brain frying environment variable configurations I've had to deal with which no permies could tell me anything about defies belief.

> someone comes and tells that it's bad and considered harmful,

Yes, some things are bad and are actively harmful. Famously, unstructured programming using gotos. Would you like to go back to that? Believe me, you would not. But perhaps you are not a programmer?

> That thing wouldn't be a de-facto standard if it was too bad, right?

It's not a "de-facto standard", it's simply bad.

Re: Don’t use environment variables for configuration

#100

Post by c0l0 »

_y5hn wrote: post #53
yokaze wrote: post #43

Earlier quoted context omitted.

File permissions allow finer granularity of access control. Environment variables are visible to any user in the system.

Not in any multi-user multi-process OS. You set environment variables in a process (ie. shell/CMD.EXE) and spawn child process (the program) from that parent. The environment variables will only be visible to those two processes.

Linux disagrees; try

    strings /proc/*/environ
to see for yourself.

On Solaris/SunOS, you could use `pargs -e $PID`. And so on.

Having separate UIDs to run your processes A and B under shields either one from peeking at the other's environment, though. UNIX DAC is simple and powerful enough for MOST security concerns, I would argue.

Post reply on HN