Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dear GitHub: no YAML anchors, please

blog.yossarian.net

Post reply on HN
Print view View original

81–90 of 159 posts

Re: Dear GitHub: no YAML anchors, please

#81

Post by woodruffw »

VGHN7XDuOXPAzol wrote: post #78
woodruffw wrote: post #71

Earlier quoted context omitted.

> This is something that a custom parser library could figure out, no? The same as how you have format-preserving TOML libraries, for instance. Yes, it's just difficult. The point made in the post isn't that it's impossible, but that it significantly changes the amount of of "ground work" that static analysis tools have to do to produce useful results for GitHub Actions. > I think it makes way more sense for GitHub t…

> conformance with YAML maybe, but not entirely sure. 'Two wrongs don't make a right' kind of thinking on my side here. But if they call it GFY and do what they want, then that would probably be better for everyone involved. > they don't have the "Norway" problem because they intentionally don't apply the boolean value rules from YAML. I think this is YAML 1.2. I have not done or seen a breakdown to see if GitHub is…

> But if they call it GFY and do what they want, then that would probably be better for everyone involved.

Yes, agreed.

> I think this is YAML 1.2. I have not done or seen a breakdown to see if GitHub is aiming for YAML 1.2 or not but they appear to think that way, given the discussion around merge keys

I think GitHub has been pretty ambiguous about this: it's not clear to me at all that they intend to support either version of the spec explicitly. Part of the larger problem here is that programming language ecosystems as a whole don't consistently support either 1.1 or 1.2, so GitHub is (I expect) attempting to strike a happy balance between their own engineering goals and what common language implementations of YAML actually parse (and how they parse it). None of this makes for a great conformance story :-)

> (though it's still not clear why flattening the YAML would not be sufficient for a static analysis tool. If the error report references a key that was actually merged out, I think users would still understand the report; it's not clear to me that's a bad thing actually)

The error report includes source spans, so the tool needs to map back to the original location of the anchor rather than its unrolled document position.

(This is table stakes for integration with formats like SARIF, which expect static analysis results to have physical source locations. It's not good enough to just say "there's a bug in this element and you need to find out where that's introduced," unfortunately.)

Re: Dear GitHub: no YAML anchors, please

#82

Post by nirvdrum »

baq wrote: post #57
nirvdrum wrote: post #41

Earlier quoted context omitted.

Having used CI systems and application frameworks that support YAML anchors for configuration, adding in a programming language would be a massive amount of complexity for very little gain. We're not talking about dozens of locations with hundreds of lines of shared code. Asking the team to add a new build dependency, learn a new language, and add a new build step would create considerably more problems, not fewer. U…

Your team doesn't know YAML, it knows github actions. There's zero transferable knowledge when switching from github actions to kubernetes deployments, as there is precisely the same zero correlation between kubernetes and ansible configs. 'It's all YAML' is a lie and I'm continuously surprised so many people are falling for it for so long. YAML is the code-as-data, but the interpreter determines what it all means.

Oh, for goodness' sake. We know YAML syntax and that's the only part that's relevant here. Pointing out that different software uses different keys for their configuration or even takes different actions for keys that happen to share the same name isn't particularly insightful. We haven't been bamboozled.

Re: Dear GitHub: no YAML anchors, please

#83

Post by anttiharju »

TheDong wrote: post #14
anttiharju wrote: post #7

I think YAML anchors in GitHub Actions are very welcome, for example for DRYing the push/pull_request 'paths:' filters [1]. Now only if they supported paths filter for `workflow_call` [2] event in addition to push/pull_request and my life would be a lot easier. Nontrivial repos have an unfortunate habit of building some sort of broken version of change detection themselves. The limit of 20 unique workflow calls is qu…

Wanna DRY out your github actions yaml? Generate it from Dhall, or cue, or python, or some real language that supports actual abstractions. If your problem is you want to DRY out yaml, and you use more yaml features to do it, you now have more problems, not fewer.

Or just use composite actions, it's not 2020 anymore.

Templating GitHub Actions is very powerful (I've worked with such a setup) but it has its own headaches and if you don't _need_ custom tooling better to not have it.

I can wish for improvements on the native setup without reaching out for the sledgehammer.

Re: Dear GitHub: no YAML anchors, please

#84

Post by TheDong »

true_blue wrote: post #61
phanimahesh wrote: post #46

Earlier quoted context omitted.

This. So true. Yaml has always been an overly complicated format, with weird quirks ( like norway becoming false in a list of country codes ). I find it an absolute shame that languages like Dhall did not become more popular earlier. Now everything in devops is yaml, and I think many developers pick yaml configs not out of good reasons but defaulting to its ubiquity as sufficient.

>norway yaml 1.2 was released in 2009, and it fixed this problem. this is an implementation issue. https://yaml.org/spec/1.2.2/#12-yaml-history

If your data format is so complicated that all commonly used implementations are not compliant with your spec, maybe it's a problem with the data-format.

Every single implementation people actually use seems to be a messy mix of yaml 1.1 and 1.2....

Maybe if the yaml project wants to consider this fixed, they should have written some correct reference parsers themselves for any languages in need, and encouraged their use.

Re: Dear GitHub: no YAML anchors, please

#85

Post by anttiharju »

anttiharju wrote: post #83
TheDong wrote: post #14

Earlier quoted context omitted.

Wanna DRY out your github actions yaml? Generate it from Dhall, or cue, or python, or some real language that supports actual abstractions. If your problem is you want to DRY out yaml, and you use more yaml features to do it, you now have more problems, not fewer.

Or just use composite actions, it's not 2020 anymore. Templating GitHub Actions is very powerful (I've worked with such a setup) but it has its own headaches and if you don't _need_ custom tooling better to not have it. I can wish for improvements on the native setup without reaching out for the sledgehammer.

I think most of the pain with GitHub Actions goes away if you use actionlint, action-validator, prettier/yamlfmt in a single ci job to validate your setup. Can even add them as git hooks that automatically stage changes and give quick feedback when iterating.

Re: Dear GitHub: no YAML anchors, please

#86

Post by woodruffw »

saurik wrote: post #79
woodruffw wrote: post #33

Earlier quoted context omitted.

It is specifically leveled at YAML anchors in GitHub. I don't have a super strong opinion of YAML anchors in other contexts. (This post is written from my perspective as a static analysis tool author. It's my opinion from that perspective that the benefits of anchors are not worth their costs in the specific context of GitHub Actions, for the reasons mentioned in the post.)

"YAML" should mean something . When I saw GitHub Actions supported "YAML", I thought "OK, certainly not my favorite, but I can deal with that ", and so I read the YAML specification, saw anchors, and then had to realize the hard way that they didn't work on GitHub Actions, leaving me unsure what even would or wouldn't work going forward. Is this even the only way it differs? I don't know, as they apparently don't use…

> "YAML" should mean something.

Sure, agreed. Another comment notes that GitHub probably should call this their own proprietary subset of YAML, and I wouldn't object to that.

> Well, hopefully they also prioritize fixing that?

I expect they won't, since it's not clear what version of YAML they even aim to be compatible with.

However, I don't understand why engineers who wouldn't jump off of a bridge because someone told them to would follow a spec to the dot just because it exists. Specifications are sometimes complicated and bad, and implementing a subset is sometimes the right thing to do!

GitHub Actions, for example, doesn't make use of the fact that YAML is actually a multi-document format, and most YAML libraries don't gracefully handle multiple documents in a single YAML stream. Should GitHub Actions support this? It's entirely unclear to me that there would be any value in them doing so; subsets are frequently the right engineering choice.

Re: Dear GitHub: no YAML anchors, please

#87

Post by theamk »

nirvdrum wrote: post #41
TheDong wrote: post #14

Earlier quoted context omitted.

Wanna DRY out your github actions yaml? Generate it from Dhall, or cue, or python, or some real language that supports actual abstractions. If your problem is you want to DRY out yaml, and you use more yaml features to do it, you now have more problems, not fewer.

Having used CI systems and application frameworks that support YAML anchors for configuration, adding in a programming language would be a massive amount of complexity for very little gain. We're not talking about dozens of locations with hundreds of lines of shared code. Asking the team to add a new build dependency, learn a new language, and add a new build step would create considerably more problems, not fewer. U…

> not talking about dozens of locations with hundreds of lines of shared code.

:) :) :)

.github/workflows in my current project: 33 files, 3913 lines total, 1588 lines unique.

(and this was _after_ we moved all we can into custom actions and sub-workflows)

Re: Dear GitHub: no YAML anchors, please

#88

Post by kiitos »

saurik wrote: post #79
woodruffw wrote: post #33

Earlier quoted context omitted.

It is specifically leveled at YAML anchors in GitHub. I don't have a super strong opinion of YAML anchors in other contexts. (This post is written from my perspective as a static analysis tool author. It's my opinion from that perspective that the benefits of anchors are not worth their costs in the specific context of GitHub Actions, for the reasons mentioned in the post.)

"YAML" should mean something . When I saw GitHub Actions supported "YAML", I thought "OK, certainly not my favorite, but I can deal with that ", and so I read the YAML specification, saw anchors, and then had to realize the hard way that they didn't work on GitHub Actions, leaving me unsure what even would or wouldn't work going forward. Is this even the only way it differs? I don't know, as they apparently don't use…

> and so I read the YAML specification

but there isn't a single YAML spec, there are at least 2 in common use: yaml 1.1, and 1.2, which have discrete specs and feature-sets. re: anchor stuff specifically, 1.1 supports merge keys whereas 1.2 explicitly does not, so that's one thing

and github actions does not actually specify which yaml spec/version it uses when parsing workflow yaml files

it's unfortunately just not the case that "YAML means something" that is well-defined in the sense that you mean here

Re: Dear GitHub: no YAML anchors, please

#89

Post by baq »

nirvdrum wrote: post #82
baq wrote: post #57

Earlier quoted context omitted.

Your team doesn't know YAML, it knows github actions. There's zero transferable knowledge when switching from github actions to kubernetes deployments, as there is precisely the same zero correlation between kubernetes and ansible configs. 'It's all YAML' is a lie and I'm continuously surprised so many people are falling for it for so long. YAML is the code-as-data, but the interpreter determines what it all means.

Oh, for goodness' sake. We know YAML syntax and that's the only part that's relevant here. Pointing out that different software uses different keys for their configuration or even takes different actions for keys that happen to share the same name isn't particularly insightful. We haven't been bamboozled.

I don’t understand even more now. If you freely admit they’re different languages, the only reason to keep using the stupid deficient syntax is momentum, and while it isn’t a bad reason, it is costing you and everyone else in the long run.

Re: Dear GitHub: no YAML anchors, please

#90

Post by nirvdrum »

WorldMaker wrote: post #74
nirvdrum wrote: post #41

Earlier quoted context omitted.

Having used CI systems and application frameworks that support YAML anchors for configuration, adding in a programming language would be a massive amount of complexity for very little gain. We're not talking about dozens of locations with hundreds of lines of shared code. Asking the team to add a new build dependency, learn a new language, and add a new build step would create considerably more problems, not fewer. U…

Use the language you are already working in? Most languages have good YAML serialization and I think in most languages a function call taking a couple parameters that vary to produce slightly different but related objects is going to be as readable or more readable than YAML anchors.

That would be better, but it's an option I already have available to me and it's just not attractive. AFAIK, GitHub Actions requires the config files to be committed. So, now I need to guard against someone making local modifications to a generated file. It's doable of course, but by the time I've set all this up, it would have been much easier for everyone to copy and paste the six lines of code in the three places they're needed. YAML anchors solve that problem without really creating any new ones.

If generating your GitHub Actions config from a programming language works for you, fantastic. I'm just happy we now have another (IMHO, attractive) option.

Post reply on HN