Common Expression Language

"Executable" configuration languages (most "non-executable" configuration language parsers are push down automatons that execute the configuration) are handy, but without strong coding standards and good discipline, the line between business logic and configuration tends to blur over time.

Cartesian product, map, and reduce operations over finite sets and lists are really handy in configuration. ("For each server in SetA look at each path in SetB and ...") But, if you find yourself starting to write general loops (as opposed to loops implementing map, reduce, and Cartesian product in languages that don't have them built-in), it's a sign you're starting to blur the line between configuration and business logic.

Unit-testing configurations is difficult, especially if they can be non-deterministic (depend on data/time/random()) and aren't modular.

In some sense, all programs with configuration files are really interpreters for the language of their configuration files. (As mentioned before, many of these abstract machines are just push down automata.) Taken too far, the configuration becomes the real program.

I've seen a (now retired) automated trading system with a powerful XML-based configuration language where a few times people got themselves into trouble (and caused trading losses) when their complex tower of configuration fell over. Part of the problem was there existed a few people who weren't trusted to write application logic, but who were trusted to "just update configurations". When the only tool some of your people are allowed to use is a hammer, hammer marks start mysteriously showing up everywhere. Additionally, this was over 10 years ago, and prior to these trading losses, configuration underwent less stringent review. I don't think my experience was atypical.

I've also seen configuration loading get stuck because someone added some code to the config to hit a REST endpoint in the middle of the configuration file. Ideally, you'd leave any I/O to the main program logic, where it's easier to perform the I/O asynchronously, or otherwise non-blocking.

Deterministic non-Turing-complete immutable "executable" configuration languages (or at least ones where it's difficult to get unbounded recursion) tend to be a happy medium. Also, declarative rather than imperative configuration languages tend to be easier to read.

Back when I was a developer in web search infra at Google, I vaguely remember once or twice using a language (maybe Borg's config language, borgconfig) that completely lacked mutability and essentially used object prototyping (A is created as a copy of B, with differences specified at object creation time.)

Every major investment bank has an in-house variation of this, commonly called “payoff language” or “cashflow language”.

> Ideological answer: For the same reason HTTP/2.0’s binary protocol didn’t instantly obviate/deprecate HTTP/1.0’s text protocol. Text has advantages: text is debuggable, and prototypable. If the interface between two programs is a text based declarative language, you can audit that text, diff that text, edit that text to see how changes affect the result, mock one side or the other by producing or consuming that text, etc.

I can see the argument for using a textual format (although I think it's weaker than you say; if we're generating this config with code then we don't want to diff or edit the generated config), but YAML seems like a singularly poor choice if you want reliable diffs and editing; it's like picking tag-soup HTML. Straight JSON (ideally with a schema), TOML or even XML seems like a better bet if you're generating it programmatically.

> And, obviously, if you don’t control the other end, you don’t decide how the other end does its config.

Right, in that case it's all moot. I took GP to be talking about what formats these tools should use. IMO if the tool is intended to consume a machine-generated config then it would be better to use a machine-oriented config format. I think the option of something like protobuf (which is language-independent) is underappreciated, but even restricting ourselves to textual options, something stricter than YAML seems like a better bet.

I use both in different projects. OPA (and its language Rego) is a good matching and policy engine with declarative blocks, modules, and expressive functions for HTTP headers, JWTs, etc. It's great for security. For building up abstractions and testing arbitrary JSON with complex, pre-defined policies. Testing is built in, and that's great. You can create "functions" and your own DSL for matching/evaluation.

If you want ABAC, use OPA. In every case.

CEL, and CEL-GO, is entirely different. It allows you to evaluate arbitrary expressions with random data. Think a search (eg. linkedin API's crappy search, or log searching, or random predicates).

You would not define complex policies in CEL like you would in OPA. Well, I would not - you can define arbitrary macros and functions in CEL but it is not made for that scale. OPA is more suited for that.

Some examples:

- In OPA, you can define a policy that matches RBAC, ownership/acl, and ABAC in one file. With multi-tenancy. Think: "as a patient, I can see my data", and "as the patient's guardian, if they're under 18, I can see their data". And "As a doctor in the patient's clinic, I can see their data". And "as a clinical director in sudo mode, I can see their data". All in the same policy package, with tests.

- OPA supports "partial evaluation". For example, if you only have a subset of data available, you can evaluate an OPA policy and have OPA tell you whether the policy evaluates to true or what data is missing. This is quite powerful for building up complex auth layers.

- In CEL, you can say "all users > 30 days old". Simple, easy, filtering. EG, with a custom date macro, `date(users.created_at) > duration("30d")`.

In short, use both. OPA for security and complex policies. CEL for user-defined "expressions".

They also have https://github.com/google/jsonnet and https://cue.googlesource.com/cue.