Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dear GitHub: no YAML anchors, please

blog.yossarian.net

Post reply on HN
Print view View original

111–120 of 159 posts

Re: Dear GitHub: no YAML anchors, please

#111

Post by freeplay »

rickette wrote: post #101
pacoWebConsult wrote: post #12

Can YAML go away entirely and instead allow pipelines to be defined with an actual language? What benefits does the runner-interpreted yaml-defined pipeline paradigm actually achieve? Especially with runners that can't be executed and tested locally, working with them is a nightmare.

GitHub Actions originally supported HCL (Hashicorp Configuration Language) instead of YAML. But the YAML force was too strong: https://github.blog/changelog/2019-09-17-github-actions-will... .

If you have worked with HCL in any serious capacity, you'll be happy they didn't go that route.

Here's some fun examples to see why HCL sucks:

- Create an if/elseif/else statement

- Do anything remotely complex with a for loop (tip: you're probably going to have to use `flatten` a lot)

Re: Dear GitHub: no YAML anchors, please

#112

Post by freeplay »

delusional wrote: post #52
jbjbjbjb wrote: post #38

Earlier quoted context omitted.

I couldn’t agree more. I think we should just write our pipelines in languages our teams are familiar with and prioritise being able to run them locally.

> prioritise being able to run them locally. That is the key function any serious CI platform needs to tackle to get me interested. FORCE me to write something that can run locally. I'll accept using containers, or maybe even VMs, but make sure that whatever I build for your server ALSO runs on my machine. I absolutely detest working on GitHub Actions because all too often it ends up requiring that I create a new rep…

+1

Working with ADO pipelines is painful.

- Make change locally

- Push change

- Run pipeline

- Wait forever because ADO is slow

- Debug the error caused by some syntax issue in their bastardized version of yaml

- Repeat

Re: Dear GitHub: no YAML anchors, please

#113

Post by Arcuru »

herpdyderp wrote: post #97
time4tea wrote: post #95

Earlier quoted context omitted.

Have you seen https://typeflows.io/ It takes your programming language version and turns it into github actions yaml, so you dont need to do any of that sort of thing.

It has pricing tiers!? That's crazy, just use https://www.npmjs.com/package/github-actions-workflow-ts

Its "pricing tiers" are "always free for OSS" and "TBD" for commercial use.

I like the things I depend on to actually have a funding model, so that's actually more appealing to me than something fully free.

Re: Dear GitHub: no YAML anchors, please

#114

Post by WorldMaker »

nirvdrum wrote: post #90
WorldMaker wrote: post #74

Earlier quoted context omitted.

Use the language you are already working in? Most languages have good YAML serialization and I think in most languages a function call taking a couple parameters that vary to produce slightly different but related objects is going to be as readable or more readable than YAML anchors.

That would be better, but it's an option I already have available to me and it's just not attractive. AFAIK, GitHub Actions requires the config files to be committed. So, now I need to guard against someone making local modifications to a generated file. It's doable of course, but by the time I've set all this up, it would have been much easier for everyone to copy and paste the six lines of code in the three places…

Most of the debate here is that a lot of us don't find YAML anchors attractive. It can be one of the papercuts of using YAML.

I mostly agree with the article that with GitHub Actions specifically, I try to refactor things to the top-level "workflow" level first, and then yeah resort to copy and paste in most other cases.

I'm a little less adamant that GitHub should remove anchor support again than the original poster, but I do sympathize greatly, having had to debug some CircleCI YAML and Helm charts making heavy use of YAML anchors. CircleCI's YAML is so bad I have explored options to build it with a build process. Yeah, it does create new problems and none of those explorations got far enough to really improve the process, but one of the pushes to explore them was certainly that YAML anchors are a mess to debug, especially when you've got some other tool concatenating YAML files together and can result in anchor conflicts (and also other parts of the same YAML that depend on a particular form of how anchor conflicts overwrite each other, oof). I don't see GitHub Actions necessarily getting that bad just by enabling anchors, but I have seen enough of where anchors become a crutch and a problem.

Re: Dear GitHub: no YAML anchors, please

#115

Post by jayd16 »

ok123456 wrote: post #106
jayd16 wrote: post #60

Earlier quoted context omitted.

Why do we think an arbitrary language is easier to reason about? If it was so easy you could just do it now. The yaml could be extremely simple and just call into your app, but most don't bother. I'm certainly willing to believe that yaml is not the ideal answer but unless we're comparing it to a concrete alternative, I feel like this is just a "grass is always greener" type take.

You don't have to reason about them. You write a compiler that enforces stronger invariants above and beyond everything is an array/string/list/number/pointer. Good general-purpose programming languages provide type systems that do just this. It is criminal that the industry simply ignores this and chooses to use blobs of YAML/JSON/XML with disastrous results---creating ad-hoc programming languages without a typesyst…

The real issue isn't the code part. You can just call into whatever arbitrary thing you want for the actual script part.

YAML is used for the declarative part of structuring the job graph. The host (in this case, GitHub) would need to call into your code to build the job graph. Which means it would need to compile your code, which means it needs its own build step. This means it would need to run on a build machine that uses minutes because GitHub is not going to just run arbitrary code for free.

There's no guarantee that your arbitrary language is thread safe or idempotent so it can't really run in parallel like how a declarative file could be used.

So now you're in a situation where you add another spin up and tear down step even if your actual graph gen call is zero cost.

There's a reason it works the way it does.

Re: Dear GitHub: no YAML anchors, please

#116

Post by biimugan »

pacoWebConsult wrote: post #12

Can YAML go away entirely and instead allow pipelines to be defined with an actual language? What benefits does the runner-interpreted yaml-defined pipeline paradigm actually achieve? Especially with runners that can't be executed and tested locally, working with them is a nightmare.

I agree somewhat with the proposition that YAML is annoying for configuring something like a workflow engine (CI systems) or Kubernetes. But having it defined in YAML is actually preferable in an enterprise context. It makes it trivial to run something like OPA policy against the configuration so that enterprise standards and governance can be enforced.

When something is written in a real programming language (that doesn't just compile down to YAML or some other data format), this becomes much more challenging. What should you do in that case? Attempt to parse the configuration into an AST and operate over the AST? But in many programming languages, the AST can become arbitrarily complex. Behavior can be implemented in such a way as to make it difficult to discover or introspect.

Of course, YAML can also become difficult to parse too. If the system consuming the YAML supports in-band signalling -- i.e. proprietary non-YAML directives -- then you would need to first normalize the YAML using that system to interpret and expand those signals. But in principal, that's still at least more tractable than trying to parse an AST.

Re: Dear GitHub: no YAML anchors, please

#117

Post by anon7000 »

Anchors are so, so useful. Buildkite (which has its own CI pipelines syntax) is a good example. Let’s say I want every pipeline step to run on my custom agents (on my self-hosted infra). I could either copy/paste an identical “agents” property across however many hundreds or thousands of CI steps I have.

Or I could use a YAML anchor.

Re: Dear GitHub: no YAML anchors, please

#118

Post by tarkaTheRotter »

herpdyderp wrote: post #97
time4tea wrote: post #95

Earlier quoted context omitted.

Have you seen https://typeflows.io/ It takes your programming language version and turns it into github actions yaml, so you dont need to do any of that sort of thing.

It has pricing tiers!? That's crazy, just use https://www.npmjs.com/package/github-actions-workflow-ts

Hey - Typeflows maintainer here. We know that there are other similar libraries out there that do some of the same thing as Typeflows, but am hoping to go much much further than anything out there to help out teams struggling with their pipelines. Examples of things on the roadmap:

- workflow visualisations (this is already working - you can see an example of workflow relationship and breakdowns on a non-trivial example at https://github.com/http4k/http4k/tree/master/.github/typeflo...);

- running workflows through an event simulator so you can tell cause and effect when it comes to what triggers what; - security testing on workflows - to avoid the many footguns that there are in GHA around secrets etc;

- compliance tests around permitted Action versions;

- publishing of reusable repository files as binary dependencies that can be upgraded and compiled into your projects - including not just GHA actions and workflows but also things like version files, composable Copilot/Claude/Cursor instruction files;

- GitLab, CircleCI, Bitbucket, Azure DevOps support using the same approach and in multiple languages;

Lots to do - and hopefully it will be valuable enough for commercial companies to pay for!

:)

Re: Dear GitHub: no YAML anchors, please

#119

Post by true_blue »

tkfu wrote: post #93
true_blue wrote: post #61

Earlier quoted context omitted.

>norway yaml 1.2 was released in 2009, and it fixed this problem. this is an implementation issue. https://yaml.org/spec/1.2.2/#12-yaml-history

I would not say it "fixed" the problem. It removed the _recommendation_ for parser implementations to use the regex `y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF` for parsing scalars as bools, it changed the _canonical_ presentation of bools from `y|n` to `true|false`, and it introduced the "schema" concept. It also introduced and recommended the use of the Core Schema, which use…

the recommendation was what caused the norway problem. it now strongly recommends not to do this, and it says that a Yaml parser should use the core schema unless instructed otherwise. going against the recommendation while saying that you're yaml 1.2 compliant feels like an issue that should be raised with the parser to me. I've never run into this issue in practice though.

is there a parser that says that it's Yaml 1.2 compliant that uses that regex? I don't know of one.

Re: Dear GitHub: no YAML anchors, please

#120

Post by tarkaTheRotter »

Levitating wrote: post #96
time4tea wrote: post #95

Earlier quoted context omitted.

Have you seen https://typeflows.io/ It takes your programming language version and turns it into github actions yaml, so you dont need to do any of that sort of thing.

Website barely loads on my old phone and I can't see any examples of the syntax.

Hey - maintainer here. Sorry about your bad experience and thanks for mentioning it! The Core Web Vitals test did come back ok - but evidently there's more to do so will get that sorted. (Web design not a strong point! ). The code examples should be showing on smaller screens when in landscape on mobile (they looked awful in portrait) - but will also look at that as well!

Could I possibly ask you to reply with the model of your phone so can make sure it works ok after have fixed?

Post reply on HN