Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YAML: Probably not so great after all

arp242.net

Post reply on HN
Print view View original

421–430 of 457 posts

Re: YAML: Probably not so great after all

#421

Post by ivalm »

norcalli wrote: post #267
jchw wrote: post #181

The issue is, I think most people (myself included) enter YAML into their lives as basically a JSON alternative with lighter syntax. Without really realizing, or perhaps without internalizing, the rather ridiculous number of different ways to represent the same thing, the painful subtle syntax differences that lead to entirely different representations, the sometimes difficult to believe number of features that the l…

I've never understood this. JSON is really not that difficult to work with manually. I tend to write my config files as JSON for utilities I write. What is it with peoples' innate aversion to braces?

Comments, comments, comments.

Seriously, our batch jobs for better or worse have configs with a bunch of parameters that are passed around as json, and while most variable names are intuitive and there is documentation on the wiki, and most often the config can be autogenerated by other tools it would still be better if when I manually open it in the config itself I would easily see the difference between n_run_threads vs n_reg_threads, etc...

Re: YAML: Probably not so great after all

#422

Post by SomeOldThrow »

DonHopkins wrote: post #420
SomeOldThrow wrote: post #415

Earlier quoted context omitted.

Why are you putting logic in the config in the first place? Just let it be data.

That's like saying that God existed before creating the universe. Then you have to ask who created God? And if God was created only from data and not any logic, then the config file must have been really huge and unmaintainable.

Do you have an example of when logic would help a configuration file? Their value and use are entirely dependent on the context.

Also I can't make a head or tail of your statement, no offense intended—I can make out parts, but the whole just has no meaning to me.

Re: YAML: Probably not so great after all

#423

Post by mjevans »

ridiculous_fish wrote: post #117
mjevans wrote: post #108

Earlier quoted context omitted.

(suggestion) Drop the 4th requirement. Having to close contexts is a VERY good 'sanity check' to see if something is malformed or not. If appending is necessary make the parser handle multiple copies of the namespace and merge them upon output. Unknown keys and sections should also always be copied from input to output (this is how you embed comments).

I'm interested, can you explain more about the merging idea? To clarify the requirement, history could be a JSON array of objects: [ {"cmd": "git checkout", "when": 1234 }, {"cmd": "vagrant up", "when": 4567 } ] To append an entry to this file and keep it valid, one must locate the closing square bracket and overwrite it. That work is what I hope to avoid.

Loosely running with your example:

A file already exists with this content:

[{"cmd": "git checkout", "when": 1234 }]

Another tool wants to add a setting/element/etc, and simply creates a new config object with only the change in question included and APPENDS the existing config.

[{"cmd": "git checkout", "when": 1234 }][{"cmd": "vagrant up", "when": 4567},{"comment": "Comments, notes, etc are kept even if they don't validate to the recognized configuration options."}]

A configuration validator / etc loads the config and merges these in state-machine order, over-writing existing values with the latest ones from the end of the stream of objects and then determining if the result is a valid configuration. (Maybe file references fail to resolve / don't open / there's some combination of settings that's not supported...)

    [
      {"cmd": "git checkout", "when": 1234 },
      {"cmd": "vagrant up", "when": 4567},
      {"comment": "... actually kept as above"}
    ]

Re: YAML: Probably not so great after all

#424

Post by DonHopkins »

SomeOldThrow wrote: post #422
DonHopkins wrote: post #420

Earlier quoted context omitted.

That's like saying that God existed before creating the universe. Then you have to ask who created God? And if God was created only from data and not any logic, then the config file must have been really huge and unmaintainable.

Do you have an example of when logic would help a configuration file? Their value and use are entirely dependent on the context. Also I can't make a head or tail of your statement, no offense intended—I can make out parts, but the whole just has no meaning to me.

Given any non-trivial data-only config file, it will always grow to the point that you'll end up needing to generate it automatically with logic. And that goes double for God's config file.

Re: YAML: Probably not so great after all

#425

Post by SomeOldThrow »

DonHopkins wrote: post #424
SomeOldThrow wrote: post #422

Earlier quoted context omitted.

Do you have an example of when logic would help a configuration file? Their value and use are entirely dependent on the context. Also I can't make a head or tail of your statement, no offense intended—I can make out parts, but the whole just has no meaning to me.

Given any non-trivial data-only config file, it will always grow to the point that you'll end up needing to generate it automatically with logic. And that goes double for God's config file.

Do you have an example? I don't think I've ever needed to generate a config automatically (except in niche cases like generating configs for services in chef), and if I'm understanding correctly, nothing proposed in this thread would help with that scenario.

I suspect you're front-loading a lot of logic from the app bootup into the config file, and I'm not sure what you stand to gain conflating those two things.

Like—what's the execution order of a config file? Can you refer to a value later in the file? If so, how does it determine which value is executed before the next? If not, how do you setup circular dependencies—can you redefine config values halfway through a file? If not, you're gonna have to fall back on at-boot pre-processing anyway with sufficient complexity, so just treat a config like dumb data to begin with and do all your logic in the bootup and put all the values/whatever in the config file. And god knows, I would be strongly tempted to murder an engineer that introduced a config file capable of rewriting itself—that engineer has clearly never needed to debug another person's shitty code before.

Re: YAML: Probably not so great after all

#426

Post by DonHopkins »

SomeOldThrow wrote: post #425
DonHopkins wrote: post #424

Earlier quoted context omitted.

Given any non-trivial data-only config file, it will always grow to the point that you'll end up needing to generate it automatically with logic. And that goes double for God's config file.

Do you have an example? I don't think I've ever needed to generate a config automatically (except in niche cases like generating configs for services in chef), and if I'm understanding correctly, nothing proposed in this thread would help with that scenario. I suspect you're front-loading a lot of logic from the app bootup into the config file, and I'm not sure what you stand to gain conflating those two things. Like…

I've developed programs with tens of thousands of lines of expanded JSON in their config files. No fucking way I'm maintaining all that by hand as pure data.

See: https://news.ycombinator.com/item?id=20735231

Also: https://en.wikipedia.org/wiki/Don%27t_repeat_yourself and https://en.wikipedia.org/wiki/Single_source_of_truth

The opposite of DRY (Don't Repeat Yourself) is WET (Write Everything Twice or We Enjoy Typing or Waste Everyone's Time) -- but twice could be ten times or more, there's no limit. Writing it all out again and again by hand as pure literal data, and hoping I didn't make any typos or omissions in each of the ten repetitions, without some sort of logical algorithmic compression and abstraction, would be idiotic.

Re: YAML: Probably not so great after all

#428

Post by DrJosiah »

SomeOldThrow wrote: post #425
DonHopkins wrote: post #424

Earlier quoted context omitted.

Given any non-trivial data-only config file, it will always grow to the point that you'll end up needing to generate it automatically with logic. And that goes double for God's config file.

Do you have an example? I don't think I've ever needed to generate a config automatically (except in niche cases like generating configs for services in chef), and if I'm understanding correctly, nothing proposed in this thread would help with that scenario. I suspect you're front-loading a lot of logic from the app bootup into the config file, and I'm not sure what you stand to gain conflating those two things. Like…

This thread hurts so bad.

I never used helm / kubernetes before 3 months ago.

Not 2 weeks ago I needed to loop in a helm config file in order to basically say "all this same config, libraries, etc., just run this other command instead" ... because someone who makes those decisions had ~100 lines of environment-injected configuration + boilerplate in the yaml that I couldn't get rid of, needed, and would have otherwise needed to copy / paste.

Since then, those environment variables have been pulled out into a different file (refactoring!), and now we replaced a loop over 100 lines of config, with 2x sets of 15-20 lines of config boilerplate. Better, but still a lot of bull. I don't know what the right answer is, because we've got less helm templating bullshit in there, but we still need boilerplate. Because it's not like I can tear down an entire kubernetes + helm infrastructure because I don't like how the config files are written.

Configs / config generation is hard, and generally awful. If you don't see it that way, congratulations; you're either a genius in your field, you've got not enough experience, and/or you're wrong. If you believe it's easy, and we're all missing something - please, by all means, write a book on how / why configurations aren't as hard as the rest of us say they are.

Best of luck to you.

Re: YAML: Probably not so great after all

#430

Post by SomeOldThrow »

DonHopkins wrote: post #426
SomeOldThrow wrote: post #425

Earlier quoted context omitted.

Do you have an example? I don't think I've ever needed to generate a config automatically (except in niche cases like generating configs for services in chef), and if I'm understanding correctly, nothing proposed in this thread would help with that scenario. I suspect you're front-loading a lot of logic from the app bootup into the config file, and I'm not sure what you stand to gain conflating those two things. Like…

I've developed programs with tens of thousands of lines of expanded JSON in their config files. No fucking way I'm maintaining all that by hand as pure data. See: https://news.ycombinator.com/item?id=20735231 Also: https://en.wikipedia.org/wiki/Don%27t_repeat_yourself and https://en.wikipedia.org/wiki/Single_source_of_truth The opposite of DRY (Don't Repeat Yourself) is WET (Write Everything Twice or We Enjoy Typing…

> I've developed programs with tens of thousands of lines of expanded JSON in their config files.

To what end? I don't get what you could possibly be putting in these files that could consume so much space without refactoring.

Post reply on HN