Use TOML for `.env` Files?

TBH, config is one of those Pandora's boxes - what happens if I want to do an integration test using a dev database and local docker images? what about testing after deployment to prod?

Given the myriad of configurations that can exist, including secrets management, static data etc. I would be very tempted to try to build tooling around a sqlite database, storing all your configs including static data, and update it with secrets at runtime. This way you can even remotely interface with the configuration for debug/monitoring, lint the config before commits etc.

I always treated .env files as "this is the current set of environment variables", containing just NAME="value" pairs.

If there are _different_ sets of variables, they should not be in that file, but in some different place, like the suggested settings.toml. Or perhaps use a bunch of different files in .envs/whatever.env and symlink them.

Let's not make everything complicated :)

So, for my own projects in Go lang, I created a simple library for loading configuration files, so it can load configuration either from .toml files, or from .env files or just from an environment variables.

Then you define your config like that:

  type ServerConfig struct {
    Address  string `env:"HTTP_ADDRESS"`
    Port     int    `env:"HTTP_PORT"`
    UseTLS   bool   `env:"HTTP_USE_TLS"`
    CertFile string `env:"HTTP_CERT_FILE"`
    KeyFile  string `env:"HTTP_KEY_FILE"`

    Timeout int `env:"HTTP_TIMEOUT"`
  }

  type Config struct {
    Server ServerConfig `env:"SERVER"`
  }

So, for example, with the code like this:

  if err := config.LoadToml(&cfg, tomlFileName); err != nil   {
   return nil, err
  }

  if err := config.LoadOverrides(&cfg); err != nil {
   return nil, err  
  }

It can load from TOML file, and from environment variables. This means - service can be either started from container, or run locally with .toml file.

I think .env is suitable if you’re just teeing up shell environment variables. That’s the purpose of an env file. The most common problem i have found is when you want to support things like lists and booleans. Both must be parsed.

The cool thing is, a file will always be configuration data, so decide what format works for you and your team and keep your wheels on the ground. Yaml, despite its opponents, is my first choice. I’ve never run into the problems people say the have with it, and I use it frequently to configure Docker-compose and kubernetes resources anyway.

My major pain point was that I had big JSON files inside my .env files. We use Cloud Foundry at work and it uses plenty of these for configuration.

I wrote a small Rust tool called ‘json_env’[0] to read JSON files and supply them as ENV vars to a program. I’m working on it in my free time and eventually want to also replace direnv with it. TOML and YAML support is also planned.

[0] https://github.com/brodo/json_env