Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Write Gitlab CI Pipelines in Python Code

gitlab.com

Post reply on HN
Print view View original

31–40 of 97 posts

Re: Write Gitlab CI Pipelines in Python Code

#31

Post by ViVr »

mixologic wrote: post #28
zmmmmm wrote: post #19

Another step in the endless cycle of configuration vs code. It's not an accident that we are in a deep cycle of constrained configuration languages (yaml/json/etc etc). People chose to go there because before that we had a cycle of using programming languages and people hated it, for all sorts of good reasons. Now I see we are on the way back into adding wrappers around the static config files, to turn them back into…

I think the main issue with using programming languages to run your CI/CD is that the whole point of CI/CD is to test and verify that your code works. Now you have code that may or may not be buggy, running your CI/CD so you'll have to set up CI/CD to test your test running code, ad nauseum. People might hate constrained configuration, but the idea is that you dont have to prove that what you're configuring is actual…

> Now you have code that may or may not be buggy

Doesn't your solution of replacing the possibly buggy code with configuration leave you with configuration that may or may not be buggy?

Re: Write Gitlab CI Pipelines in Python Code

#32

Post by kortex »

zmmmmm wrote: post #19

Another step in the endless cycle of configuration vs code. It's not an accident that we are in a deep cycle of constrained configuration languages (yaml/json/etc etc). People chose to go there because before that we had a cycle of using programming languages and people hated it, for all sorts of good reasons. Now I see we are on the way back into adding wrappers around the static config files, to turn them back into…

The big difference is that config files are, at some point after interpolation and templating, static data structures composed of primitive data types.

Further, these configs usually act as declarative languages, giving directives to pipelines, controllers, etc., typically with idempotent side-effects.

This is just good software engineering. You have some static data structure which describes some result you want to exist in the world, and your pipeline/program makes that outcome manifest. This is the essence of boundary programming/hexagonal architecture.

https://www.destroyallsoftware.com/talks/boundaries

Re: Write Gitlab CI Pipelines in Python Code

#33

Post by sytse »

kmstout wrote: post #12

Several weeks ago I poked around Github's CI workflow references. It seems that there's a clear path for crafting a workflow that simply grabs the text of a each issue (as it's submitted) and appends it to the documentation before closing the issue, perhaps marking it as "expected behavior". In short, an evening of tinkering could automatically turn all of a project's bugs into features for the foreseeable future. Th…

At GitLab we take the 'Release notes' section of each issue and use it to automatically generate the release post. For an example of a release post see https://about.gitlab.com/releases/2021/04/22/gitlab-13-11-re...

Re: Write Gitlab CI Pipelines in Python Code

#34

Post by encryptluks2 »

zmmmmm wrote: post #19

Another step in the endless cycle of configuration vs code. It's not an accident that we are in a deep cycle of constrained configuration languages (yaml/json/etc etc). People chose to go there because before that we had a cycle of using programming languages and people hated it, for all sorts of good reasons. Now I see we are on the way back into adding wrappers around the static config files, to turn them back into…

Template languages are meant to change a lot less than programming languages and adapt when they do. I see companies using programming languages like this and then in a year the libraries are no longer maintained, they have some churn and their new employees then want to rewrite everything. I rarely think it is worth writing CI/CD in a programming language unless you are committed to supporting it for its lifetime. Most companies will not be able to and they'll end up back at the standard templates sooner or later.

Re: Write Gitlab CI Pipelines in Python Code

#35

Post by zmmmmm »

ViVr wrote: post #31
mixologic wrote: post #28

Earlier quoted context omitted.

I think the main issue with using programming languages to run your CI/CD is that the whole point of CI/CD is to test and verify that your code works. Now you have code that may or may not be buggy, running your CI/CD so you'll have to set up CI/CD to test your test running code, ad nauseum. People might hate constrained configuration, but the idea is that you dont have to prove that what you're configuring is actual…

> Now you have code that may or may not be buggy Doesn't your solution of replacing the possibly buggy code with configuration leave you with configuration that may or may not be buggy?

Its mainly a question of it being deterministic and reproducible I think. The presumption is that you then go and test a bunch of stuff based on whatever was built, so no, it will not be buggy.

But if the code that actually goes to production is actually different to what is tested (because your rebuilt it based on a config with some runtime behavior that executed differently), you are compromising the whole thing and all bets are off.

People will try to split the difference and generate a static config with code and test that as a reproducible entity, but then you have another set of tradeoffs (not least, complexity).

Re: Write Gitlab CI Pipelines in Python Code

#36

Post by VWWHFSfQ »

zmmmmm wrote: post #19

Another step in the endless cycle of configuration vs code. It's not an accident that we are in a deep cycle of constrained configuration languages (yaml/json/etc etc). People chose to go there because before that we had a cycle of using programming languages and people hated it, for all sorts of good reasons. Now I see we are on the way back into adding wrappers around the static config files, to turn them back into…

> nobody is ever happy or at least willing to just settle for things that are "ok" but not "perfect"

I hope we never settle for things that are "ok" but not "perfect".

Re: Write Gitlab CI Pipelines in Python Code

#37

Post by izgzhen »

zmmmmm wrote: post #19

Another step in the endless cycle of configuration vs code. It's not an accident that we are in a deep cycle of constrained configuration languages (yaml/json/etc etc). People chose to go there because before that we had a cycle of using programming languages and people hated it, for all sorts of good reasons. Now I see we are on the way back into adding wrappers around the static config files, to turn them back into…

One important thing about typical config language is that they are much simpler to do static analysis upon than full fledged PL.

Static analysis, even just simple parsing, is a quite common need when you need to migrate or take statistics of something related to all configurations in a given code base.

Re: Write Gitlab CI Pipelines in Python Code

#38

Post by GongOfFour »

zmmmmm wrote: post #19

Another step in the endless cycle of configuration vs code. It's not an accident that we are in a deep cycle of constrained configuration languages (yaml/json/etc etc). People chose to go there because before that we had a cycle of using programming languages and people hated it, for all sorts of good reasons. Now I see we are on the way back into adding wrappers around the static config files, to turn them back into…

In my opinion generating configuration from Python (or any language with a robust standard library) is still a win if it means less Ops and DevOps engineers are stringing together convoluted bash scripts that torture sed/awk/jq/yq to obscene lengths.

Re: Write Gitlab CI Pipelines in Python Code

#40

Post by strogonoff »

zmmmmm wrote: post #19

Another step in the endless cycle of configuration vs code. It's not an accident that we are in a deep cycle of constrained configuration languages (yaml/json/etc etc). People chose to go there because before that we had a cycle of using programming languages and people hated it, for all sorts of good reasons. Now I see we are on the way back into adding wrappers around the static config files, to turn them back into…

The middle ground between flexibility of a programming language and constraints of a static config syntax appears to be right about where Dhall’s niche is.
Post reply on HN