In my opinion there's a "level 4.5" in between structured templating and full-blown procedural scripting: Using a general-purpose language to generate structured data, but then handing that over to a simpler system to materialise. Pulumi is the best known example. Also, any time a normal programming language is used to generate something like an ARM template or any other kind of declarative deployment file. This is t…
Levels of configuration languages
61–69 of 69 posts
Re: Levels of configuration languages
#62https://jsonnet.org/ I never heard of this before. This seems like the JSON I wish I really had. Of course at some point you could just use JavaScript. I guess that fits w option 5.
Dave Cunningham created jsonnet from some conversations I had with him about how Nix's lazy language allows one to make recursive references between parts of one's configuration in a declarative way. No need to order the evaluation beforehand. Dave also designed a way of doing "object oriented" programming in Nix which eventually turned into what is now known as overlays. P.S. I'm pretty sure jsonnet is Turing comple…
For anyone interested, this is how I'd illustrate Nix's overlays in Haskell (I know I know, I'm using one obscure lang to explain another...):
data Attr a = Leaf a | Node (AttrSet a)
deriving stock (Show, Functor)
newtype AttrSet a = AttrSet (HashMap Text (Attr a))
deriving stock (Show, Functor)
deriving newtype (Semigroup, Monoid)
type Overlay a = AttrSet a -> AttrSet a -> AttrSet a
apply :: forall a. [Overlay a] -> AttrSet a -> AttrSet a
apply overlays attrSet = fix go
where
go :: AttrSet a -> AttrSet a
go final =
let fs = map (\overlay -> overlay final) overlays
in foldr (\f as -> f as as) attrSet fs
Which uses fix to tie the knot, so that each overlay has access to the final result of applying all overlays. To illustrate, if we do: find :: AttrSet a -> Text -> Maybe (Attr a)
find (AttrSet m) k = HMap.lookup k m
set :: AttrSet a -> Text -> Attr a -> AttrSet a
set (AttrSet m) k v = AttrSet $ HMap.insert k v m
overlayed =
apply
[ \final prev -> set prev "a" $ maybe (Leaf 0) (fmap (* 2)) (find final "b"),
\_final prev -> set prev "b" $ Leaf 2
]
(AttrSet $ HMap.fromList [("a", Leaf 1), ("b", Leaf 1)])
we get: λ overlayed
AttrSet (fromList [("a",Leaf 4),("b",Leaf 2)])
Note that "a" is 4, not 2. Even though the "a = 2 * b" overlay was applied before the "b = 2" overlay, it had access to the final value of "b." The order of overlays still matters (it's right-to-left in my example tnx for foldr). For example, if I were to add another "b = 3" overlay in the middle, then "a" would be 6, not 4 (and if I add it to the end instead then "a" would stay 4).Re: Levels of configuration languages
#63Earlier quoted context omitted.
Dave Cunningham created jsonnet from some conversations I had with him about how Nix's lazy language allows one to make recursive references between parts of one's configuration in a declarative way. No need to order the evaluation beforehand. Dave also designed a way of doing "object oriented" programming in Nix which eventually turned into what is now known as overlays. P.S. I'm pretty sure jsonnet is Turing comple…
I would love to read more about all this, especially how overlays came from "object-oriented" programming. To me, the interesting part is their self-referential nature, for which lazy eval is indeed a great fit! For anyone interested, this is how I'd illustrate Nix's overlays in Haskell (I know I know, I'm using one obscure lang to explain another...): data Attr a = Leaf a | Node (AttrSet a) deriving stock (Show, Fun…
# Object Oriented Programming library for Nix
# By Russell O'Connor in collaboration with David Cunningham.
#
# This library provides support for object oriented programming in Nix.
# The library uses the following concepts.
#
# A *class* is an open recursive set. An open recursive set is a function from
# self to a set. For example:
#
# self : { x = 4; y = self.x + 1 }
#
# Technically an open recursive set is not recursive at all, however the function
# is intended to be used to form a fixed point where self will be the resulting
# set.
#
# An *object* is a value which is the fixed point of a class. For example:
#
# let class = self : { x = 4; y = self.x + 1; };
# object = class object; in
# object
#
# The value of this object is '{ x = 4; y = 5; }'. The 'new' function in this
# library takes a class and returns an object.
#
# new (self : { x = 4; y = self. x + 1; });
#
# The 'new' function also adds an attribute called 'nixClass' that returns the
# class that was originally used to define the object.
#
# The attributes of an object are sometimes called *methods*.
#
# Classes can be extended using the 'extend' function in this library.
# the extend function takes a class and extension, and returns a new class.
# An *extension* is a function from self and super to a set containing method
# overrides. The super argument provides access to methods prior to being
# overloaded. For example:
#
# let class = self : { x = 4; y = self.x + 1; };
# subclass = extend class (self : super : { x = 5; y = super.y * self.x; });
# in new subclass
#
# denotes '{ x = 5; y = 30; nixClass = ; }'. 30 equals (5 + 1) * 5).
#
# An extension can also omit the 'super' argument.
#
# let class = self : { x = 4; y = self.x + 1; };
# subclass = extend class (self : { y = self.x + 5; });
# in new subclass
#
# denotes '{ x = 4; y = 9; nixClass = ; }'.
#
# An extension can also omit both the 'self' and 'super' arguments.
#
# let class = self : { x = 4; y = self.x + 1; };
# subclass = extend class { x = 3; };
# in new subclass
#
# denotes '{ x = 3; y = 4; nixClass = ; }'.
#
# The 'newExtend' function is a composition of new and extend. It takes a
# class and and extension and returns an object which is an instance of the
# class extended by the extension.
rec {
new = class :
let instance = class instance // { nixClass = class; }; in instance;
extend = class : extension : self :
let super = class self; in super //
(if builtins.isFunction extension
then let extensionSelf = extension self; in
if builtins.isFunction extensionSelf
then extensionSelf super
else extensionSelf
else extension
);
newExtend = class : extension : new (extend class extension);
}
In nix overlays, the names "final" and "prev" used to be called "self" and "super", owing to this OOP heritage, but people seemed to find those names confusing. Maybe you can still find old instances of the names "self" and "super" in places.Re: Levels of configuration languages
#64Earlier quoted context omitted.
I would love to read more about all this, especially how overlays came from "object-oriented" programming. To me, the interesting part is their self-referential nature, for which lazy eval is indeed a great fit! For anyone interested, this is how I'd illustrate Nix's overlays in Haskell (I know I know, I'm using one obscure lang to explain another...): data Attr a = Leaf a | Node (AttrSet a) deriving stock (Show, Fun…
I have the following file called oop.nix dated from that time. # Object Oriented Programming library for Nix # By Russell O'Connor in collaboration with David Cunningham. # # This library provides support for object oriented programming in Nix. # The library uses the following concepts. # # A *class* is an open recursive set. An open recursive set is a function from # self to a set. For example: # # self : { x = 4; y…
Re: Levels of configuration languages
#65I’d argue Terraform/HCL is quite popular as a Level 4 configuration language. My biggest issue with it is that once things get sufficiently complex, you wish you were using a Level 5 language. In fact, it’s hard to see where a Level 4 language perfectly fits. After you’ve surpassed the abilities of JSON or YAML (and you don’t opt for slapping on a templating engine like Helm does), it feels like jumping straight to L…
I'm very surprised we don't see more people using a level 5 language to generate Terraform (as level 3 JSON) for this exact reason. It would seem to be the best of both worlds -- use the powerful language to enforce consistency and correctness while still being able to read and diff the simple output to gain understanding. In this hypothetical workflow, Terraform constructs like variables and modules would not be use…
Re: Levels of configuration languages
#66About 20 or so years ago, I have come across a configuration pattern that could be arguably called "Level 0". It was configuration by file existence. The file itself would be typically empty. So no parsing, syntax, or schema involved. For example, if the file /opt/foo/foo.txt exists, the software does one thing but if it is missing the software does another thing. So effectively, the existence of the file serves as a…
Re: Levels of configuration languages
#67Earlier quoted context omitted.
I have the following file called oop.nix dated from that time. # Object Oriented Programming library for Nix # By Russell O'Connor in collaboration with David Cunningham. # # This library provides support for object oriented programming in Nix. # The library uses the following concepts. # # A *class* is an open recursive set. An open recursive set is a function from # self to a set. For example: # # self : { x = 4; y…
Thanks for this! It's always nice to learn about the origins of things. I was around when they were called "self"/"super", but I never made the connection to OOP.
Re: Levels of configuration languages
#68Earlier quoted context omitted.
Only if what you specifically want is to represent queries. If what you want to represent is roughly static data then SQL is an incredibly awkward language to use.
This is valid SQL: SELECT 'Constant'
I've recently been writing quite a lot of unit tests for SQL, and the biggest pain point is setting up the state of your data, because SQL is just not an ergonomic language in which to do that. Mostly I've just ended up writing YAML describing the rows I want in a database (and I use proto fields in DBs so some nesting there), and using a utility that converts that can load that YAML as a table.
Re: Levels of configuration languages
#69Lisp code is represented in the same data structure it manipulates. This homoiconicity makes Lisp code be a great config data especially in a Lisp program. In comparison, you can't represent JS code in JSON.