Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Dhall Configuration Language

dhall-lang.org

Post reply on HN
Print view View original

181–190 of 192 posts

Re: The Dhall Configuration Language

#181

Post by Groxx »

q845712 wrote: post #110
Groxx wrote: post #77

Earlier quoted context omitted.

Have you seen Starlark? It's not too far from that, but safer in a number of ways: https://github.com/bazelbuild/starlark

I said this above as well: ytt ( https://carvel.dev/ytt/ ) lets you embed starlark into valid yaml, among other cute tricks for managing biz-logic in configs.

Huh. Clever strategy, though I'm not sure it'd be something I'd use...

It is a pretty decent "take your existing yaml and make incremental improvements" setup though. Rewriting all of your config from scratch is rarely an enjoyable experience.

Re: The Dhall Configuration Language

#182

Post by bigcat12345678 »

eru wrote: post #137
bigcat12345678 wrote: post #107

Earlier quoted context omitted.

CUE's author invented Borg configuration language or BCL since 2008. BCL code is the 3rd largest human written code in Google internal code base. In 2019 Aprial, there is 180M lines of BCL, while C++/Java sits at ~300M. BCL configuration's large scale use probably is beyond any other infra as code use cases known to human. And the learning and ideas over more than a decade, is manifested in CUE. Personally, this is e…

For the record, when I was an SRE at Google, most people I talked to hated BCL with a passion.

You are hating BCL because it's not loved by anyone. It's like the poor kid who were supporting the prodigy that is the Borg, but is actually supporting the Borg ecosystem on its back, where the prodigy kid just grab all of the acclaim.

That's just typical of almost all human involved affairs. Some dudes hard support but get 20% of what they deserve. Whereas some few get 800%.

As for BCL, it literally had no investment since 2010, yet still reliably March on ward with little support, in which case 99.99999% of software would simply die into oblivion. BCL just shows it's power and strength.

To name the importance of any single software for Google's success, the 120k lines of CPP code in borgcfg stands on a high peak that look down upon all the other dwarfs with ease.

Re: The Dhall Configuration Language

#183

Post by jonathanyc »

rtpg wrote: post #89
jonathanyc wrote: post #42

I wonder if a version of Python with types with some way to force programs to be total would be a 'good enough' substitute. JavaScript has `"use strict";`, `# use total` could ban recursion and loops over non-constants. You'd have to work hard to ensure things were really total, e.g. ban assigning functions to variables, but maybe you could make it hard enough to pay off.

Starlark is a version of this, and it's used by Bazel in particular. It's very cool tech, and sidesteps the difficulties of ensuring Python works without shipping a specific Python interpreter.

Didn't know about Starlark, that is awesome.

Re: The Dhall Configuration Language

#185

Post by hsn915 »

jordanbeiber wrote: post #120
hsn915 wrote: post #115

Relying on configuration files to glue together a huge stack of tools and services in order to produce an end product is a huge mistake. I say it's a mistake fully knowing that nearly everyone in the industry is doing it. Configuration is opaque. Undebuggable. Unmaintainable. By its very nature. No matter what "language" you use for it. You should strive to keep configurable things to an absolute minimum. Although it…

Wholeheartedly agree - convention before configuration wins with tooling even at modest scale. Larger team(s) using free form yaml configuration and templating quickly becomes messy. Setting a boundary at “configuration” and exposing configuration options to dev teams through settings works really well. It requires you to have a tooling team with a few developers, but it scales!

Convention before configuration doesn't solve the core problem.

Why do you need configuration? Because you build your application by gluing together a multitude of different applications that need to configured properly to be able to talk to each other.

Once you understand this, the solution is obvious: don't build your application this way.

Build it in one language. If you want to "reuse" existing solutions, reuse them as libraries, not as separate programs.

Re: The Dhall Configuration Language

#186

Post by hsn915 »

rad_gruchalski wrote: post #161
hsn915 wrote: post #147

Earlier quoted context omitted.

Configuration is some sort of information that is stored in a text file, for reading by one or more programs, where these programs cannot meaningfully operate without this critical information in the configuration files. If you have ever worked in the web startup industry, their use is endemic in everything. You often cannot deploy any web applciation without properly understanding tons of configuration files. Becaus…

> Because to understand the configuration files, you have to understand the programs they are inteded for and the environments they end up in, and the language they are written in. Seems like the right approach. To use something, one has to understand how to use it. Why is this unusual?

Your comment makes it sound like my problem with configurations is that they require understanding something.

But that's not what I said.

What I said is:

> Configuration is opaque. Undebuggable. Unmaintainable. By its very nature. No matter what "language" you use for it.

The underlying point - perhaps not metioned explicitly - is that they are way too complex for what you achieve with them.

Of course, complex systems require a deep level of understanding. But everything being equal, complexity is a thing to avoid - when you can do it without loss of capability.

Re: The Dhall Configuration Language

#187

Post by jordanbeiber »

hsn915 wrote: post #185
jordanbeiber wrote: post #120

Earlier quoted context omitted.

Wholeheartedly agree - convention before configuration wins with tooling even at modest scale. Larger team(s) using free form yaml configuration and templating quickly becomes messy. Setting a boundary at “configuration” and exposing configuration options to dev teams through settings works really well. It requires you to have a tooling team with a few developers, but it scales!

Convention before configuration doesn't solve the core problem. Why do you need configuration? Because you build your application by gluing together a multitude of different applications that need to configured properly to be able to talk to each other. Once you understand this, the solution is obvious: don't build your application this way. Build it in one language. If you want to "reuse" existing solutions, reuse t…

Say sql database:

Teams have agreed to use postgres for sql, this is by agreed convention.

A tooling team implement hashicorp vault and build a pg module/deployment that is packaged and accessible through whatever tooling you build.

This makes a datasource of type pg sql available to all teams in a couple of minutes.

These modules/deployments are governed by a bunch of conventions, but those are of no special concern to consuming dev teams.

Use “something” to keep track of service upstream/downstreams: consul, a database, or something else.

When a service is connected to a database, say using consul upstreams you have the job gen api inject the appropriate env vars according to an agreed upon convention.

Now dev teams have an option to deploy incredibly secure pg databases, and by simply setting a service as downstream it will automatically receive rotating connection strings.

There’s no configuration to be managed by dev teams except keeping a service graph up-to-date.

Lot of conventions above, and a tooling dev team with a lot of infra and tooling expertise.

I believe that for scaling dev teams you should look at tooling like a product domain and treat it as such.

Re: The Dhall Configuration Language

#188

Post by eru »

bigcat12345678 wrote: post #182
eru wrote: post #137

Earlier quoted context omitted.

For the record, when I was an SRE at Google, most people I talked to hated BCL with a passion.

You are hating BCL because it's not loved by anyone. It's like the poor kid who were supporting the prodigy that is the Borg, but is actually supporting the Borg ecosystem on its back, where the prodigy kid just grab all of the acclaim. That's just typical of almost all human involved affairs. Some dudes hard support but get 20% of what they deserve. Whereas some few get 800%. As for BCL, it literally had no investme…

You could use nearly the same argument to sing the praises of PHP at Facebook. (Or Cobol at banks.) Doesn't make either language any better.

Re: The Dhall Configuration Language

#189

Post by eru »

valenterry wrote: post #175
eru wrote: post #168

Earlier quoted context omitted.

Flaky in the sense that you can have stuff that's fitting right into your cut-off time limit on some runs, and going over on some others.

I mean, theoretically yes - but I doubt that this would be an issue in any but the rarest cases.

Depends on how aggressive your timeouts are, I guess?

Re: The Dhall Configuration Language

#190

Post by bigcat12345678 »

thethethethe wrote: post #123
plorkyeran wrote: post #117

Earlier quoted context omitted.

> In 2019 Aprial, there is 180M lines of BCL, while C++/Java sits at ~300M. It's... not obvious to me that that's a good thing? The ratio of configuration code to code in the things being configured being that close makes me think that BCL is something that's ill-suited to what it's now being asked to do but there's too much of it to realistically replace. Maybe it's amazing and the problem it's solving is so complic…

The number of lines of BCL at Google says little-to-nothing about the efficacy of the language itself, it's more of a reflection of the complexity and scale of the _systems_ its used to configure.

BCL is great, it's just lived out by its longevity.

But the point is that this large scale application offered the space for exploring the space of configuration as code, infra as code, and many relevant technical problem space for designing and deploying configuration.

Post reply on HN