Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The CD Pipeline Manifesto

manifesto.getglu.dev

Post reply on HN
Print view View original

21–30 of 51 posts

Re: The CD Pipeline Manifesto

#21

Post by rat87 »

> Pipeline definitions are scattered across multiple tools—GitHub Actions, Jenkins, ArgoCD, Kubernetes—and environments. This fragmentation leads to confusion, configuration drift, and duplicated effort.

So are they talking about some sort of meta language compiling into multiple yaml configs for the different environments or a single separate CI tool that has plugins and integrates with GitHub/gitlab/etc?

I do agree with them about the need for a real programming language. I hate yaml in gitlabs config, it is very hard to interpret how it will be interpreted. Things were much easier when I was scripting Jenkins even though I didn't know or like groovy then with gitlab

Re: The CD Pipeline Manifesto

#22

Post by wesselbindt »

paweladamczuk wrote: post #5

> Without types, it is difficult to compose pipelines together. I would gladly hear this argument expanded. It's really not obvious to me that that's the case.

Suppose I give you two functions f, and g. Can you run f(g()) without breaking things? The honest answer is you don't know until you read the functions, which is a slow and difficult thing to do.

Suppose I give you functions f and g of respective types int -> str and Nothing -> str. Can you compose them? No, and you see this immediately from the types. Types make reasoning about composability a lot easier.

Of course, it's not a panacea, and it's less helpful the more side effects a function has. Can we compose pure int->int functions? Of course! Can we compose two of them where the second expects some image to exist in some docker registry? You'll need to read the first to be able to tell.

Given the highly side effectful nature of pipelines, I'd think the applicability of types would be limited. But maybe that's just a lack of imagination on my part.

Certainly information like "this pipeline expects these variables" and "this pipeline sets these variables" are susceptible to a typed approach, and it would make things easier. By how much, I don't know.

Re: The CD Pipeline Manifesto

#23

Post by pdimitar »

drowsspa wrote: post #19
maccard wrote: post #16

Earlier quoted context omitted.

If I want to build and test a golang app and push it to a container repository, what value does a makefile provide over go build && docker push? All the tools do their own dependency tracking already (unfortunately).

Honestly I like it just to keep it as a command runner with the needed flags. Then in the unholy YAML there's just make build, make test, etc

The `just` tool is a better and much easier to understand command runner than `make`, however. Much less feature surface, too, which eliminates nasty surprises coming from the unnecessary complexity of `make`.

Re: The CD Pipeline Manifesto

#24

Post by jas39 »

maccard wrote: post #16
jas39 wrote: post #11

I'm thinking before we build a CI/CI pipeline, make sure there is a Makefile. Why have they gone out of style?

If I want to build and test a golang app and push it to a container repository, what value does a makefile provide over go build && docker push? All the tools do their own dependency tracking already (unfortunately).

Maybe none, but at some point you may want to do other things at buildtime, such as generating an sqlite db or generate code stubs for protobuf. Having a universal, and highly refined, tool like Make will help developers without domain knowledge. It also does not exclude the use of other tool like Just and Docker. A Makefile is also an easy jump-off point for a build pipeline.

Re: The CD Pipeline Manifesto

#25

Post by maccard »

jas39 wrote: post #24
maccard wrote: post #16

Earlier quoted context omitted.

If I want to build and test a golang app and push it to a container repository, what value does a makefile provide over go build && docker push? All the tools do their own dependency tracking already (unfortunately).

Maybe none, but at some point you may want to do other things at buildtime, such as generating an sqlite db or generate code stubs for protobuf. Having a universal, and highly refined, tool like Make will help developers without domain knowledge. It also does not exclude the use of other tool like Just and Docker. A Makefile is also an easy jump-off point for a build pipeline.

Then introduce it for those things. But a makefile to call go build, go test, docker push, Ecs update-services provides no value other than all of a sudden not working on windows without another tool.

Re: The CD Pipeline Manifesto

#26

Post by maccard »

drowsspa wrote: post #19
maccard wrote: post #16

Earlier quoted context omitted.

If I want to build and test a golang app and push it to a container repository, what value does a makefile provide over go build && docker push? All the tools do their own dependency tracking already (unfortunately).

Honestly I like it just to keep it as a command runner with the needed flags. Then in the unholy YAML there's just make build, make test, etc

At that point you end up with a makefile that has a 1:1 mapping with targets in my experience. At a previous job, we had an enormous makefile, most of which was defining phone targets and translating make arguments into maven arguments. All the actual targets were calling maven. Make provided no value at all in that, other than requiring you to know Make and maven to modify anything in the build.

Personally I’d rather a shell script for a command runner in most cases

Re: The CD Pipeline Manifesto

#27

Post by drowsspa »

maccard wrote: post #26
drowsspa wrote: post #19

Earlier quoted context omitted.

Honestly I like it just to keep it as a command runner with the needed flags. Then in the unholy YAML there's just make build, make test, etc

At that point you end up with a makefile that has a 1:1 mapping with targets in my experience. At a previous job, we had an enormous makefile, most of which was defining phone targets and translating make arguments into maven arguments. All the actual targets were calling maven. Make provided no value at all in that, other than requiring you to know Make and maven to modify anything in the build. Personally I’d rathe…

Yeah, mine often just defers back to some shell scripts. But it's useful to enumerate them in the Makefile.

Re: The CD Pipeline Manifesto

#28

Post by drowsspa »

pdimitar wrote: post #23
drowsspa wrote: post #19

Earlier quoted context omitted.

Honestly I like it just to keep it as a command runner with the needed flags. Then in the unholy YAML there's just make build, make test, etc

The `just` tool is a better and much easier to understand command runner than `make`, however. Much less feature surface, too, which eliminates nasty surprises coming from the unnecessary complexity of `make`.

There are a pletora of tools better than Make. But it's a standard tool, everyone is familiar with it, you probably don't even have to install it.

Re: The CD Pipeline Manifesto

#29

Post by pdimitar »

drowsspa wrote: post #28
pdimitar wrote: post #23

Earlier quoted context omitted.

The `just` tool is a better and much easier to understand command runner than `make`, however. Much less feature surface, too, which eliminates nasty surprises coming from the unnecessary complexity of `make`.

There are a pletora of tools better than Make. But it's a standard tool, everyone is familiar with it, you probably don't even have to install it.

`just` is 90% similar to `make` in syntax, only it has 100x less foot guns. :)

Also I'll never understand the appeal of "not having to install a tool". We're not in the 1980s anymore when that was an actual chore. You run a command, the tool is there (including in CI/CD), boom, done. What am I missing here?

The advantages you list are flimsy at best.

Re: The CD Pipeline Manifesto

#30

Post by ttyprintk »

GrumpyCat42 wrote: post #18
jas39 wrote: post #11

I'm thinking before we build a CI/CI pipeline, make sure there is a Makefile. Why have they gone out of style?

My struggle with Make and bash is that they're not very expressive - maybe that's something we want in our CIs, but I've always preferred writing an actual program in that program's native language for CI/CD, even if it has to shell out some commands every now and again.

I prefer that, too. I've heard (less-experienced) tech leads forbid Makefiles because they're not declarative enough compared to yaml.
Post reply on HN