Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beyond CI/CD: GitLab's DevOps Vision (2017)

about.gitlab.com

Post reply on HN
Print view View original

31–40 of 72 posts

Re: Beyond CI/CD: GitLab's DevOps Vision (2017)

#31

Post by tazjin »

akvadrako wrote: post #21
tazjin wrote: post #7

Slightly offtopic from what the article focuses on, but the thing that bothers me about all these built-in CI/CD things in software like Gitlab is that people seem to be perfectly content with building an artifact and then just ... pushing it out. In traditional deployments that may be some kind of "copy this thing over SSH and make it go!", in Kubernetes-land it's more like "lets just modify this API-state to point…

Sorry but it's not clear what you are saying. What is the fundamental difference between modifying your k8s deployment to point to a new tag and your solution? You can easily roll back (except with database migrations) and easily see what version is deployed when.

When you update a Kubernetes deployment in the API directly you're essentially modifying a global variable. If you do this as the result of some imperative process (e.g. a CI pipeline), you don't have any record telling you what the value should be - you only know what it currently is.

You're also only modifying a single piece of your whole state at any given time, meaning that if you have the services A and B and their deployment pipelines independently modify state - you have nothing that declares any relationship between which versions of these should be deployed together.

There's a little piece of infrastructure wisdom I've learned over the years, I refer to it sometimes as "tazjin's law":

Any infrastructure component not controlled by a reconciliation process will eventually fail.

Versions of dependent components will get out of sync, configuration is being deployed independently of the application, and so on.

In order to reconcile your current state with your desired state you must know what your desired state is.

Does that explain it?

Re: Beyond CI/CD: GitLab's DevOps Vision (2017)

#32

Post by akvadrako »

tazjin wrote: post #31
akvadrako wrote: post #21

Earlier quoted context omitted.

Sorry but it's not clear what you are saying. What is the fundamental difference between modifying your k8s deployment to point to a new tag and your solution? You can easily roll back (except with database migrations) and easily see what version is deployed when.

When you update a Kubernetes deployment in the API directly you're essentially modifying a global variable. If you do this as the result of some imperative process (e.g. a CI pipeline), you don't have any record telling you what the value should be - you only know what it currently is . You're also only modifying a single piece of your whole state at any given time, meaning that if you have the services A and B and t…

That doesn't explain it. You always need some global variable saying what the current state is. In your case, maybe it's the branch pointer in your git tree. In the k8s deployment, it's a tag value.

Re: Beyond CI/CD: GitLab's DevOps Vision (2017)

#33

Post by bpicolo »

Already__Taken wrote: post #13
FooBarWidget wrote: post #5

Earlier quoted context omitted.

Last time I checked (a few weeks ago), Gitlab's CI/CD system is great for most use cases as long as you can describe your pipeline as a list of non-interactive tasks -- some of which may be parallelized -- that each can run in a Docker container. For one of the software projects I'm involved in, I have more complex needs. We release binaries for multiple platforms so our Jenkins master delegates certain tasks to slav…

Can't you put the gitlab runner on whatever you want and set up CI environments to the runner?

The issue is that the configuration format makes complex jobs difficult to define. It's good for the simple case, but versus e.g. a Jenkinsfile[0] where you get a complete groovy DSL that can interact with it's environment across tasks, pipeline, branch, etc with ease it's rough. Having fine-grained control becomes important.

[0] https://jenkins.io/doc/book/pipeline/jenkinsfile/

Re: Beyond CI/CD: GitLab's DevOps Vision (2017)

#34

Post by tazjin »

akvadrako wrote: post #32
tazjin wrote: post #31

Earlier quoted context omitted.

When you update a Kubernetes deployment in the API directly you're essentially modifying a global variable. If you do this as the result of some imperative process (e.g. a CI pipeline), you don't have any record telling you what the value should be - you only know what it currently is . You're also only modifying a single piece of your whole state at any given time, meaning that if you have the services A and B and t…

That doesn't explain it. You always need some global variable saying what the current state is. In your case, maybe it's the branch pointer in your git tree. In the k8s deployment, it's a tag value.

No, those don't contain the same information.

Our Kontemplate repository contains the state of an entire cluster, including all versions and all configuration.

A tag value is a single piece of mutable data that is in no relation to the other relevant data.

Re: Beyond CI/CD: GitLab's DevOps Vision (2017)

#36

Post by Walkman »

Now I’m pretty sure GitLab has a focus problem. It’s already very complex with somewhat unrelated things. You can’t be everything for everybody. This will even further decrease quality of individual parts of the system. It has been unusably slow for a long time. If they try to do everything, they will have way bigger quality issues than slowness.

Re: Beyond CI/CD: GitLab's DevOps Vision (2017)

#37

Post by jsty »

adrianN wrote: post #12
foepys wrote: post #6

Earlier quoted context omitted.

Gitlab in a way suffers from being open source. Their open source product is so good and includes so many features (even a full featured CI/CD pipeline!), that they need something big to justify their paid version. So I don't blame them. You can disable a lot of stuff that eats away your memory.

Big corporations also really love bloatware because you can tick off so many feature requirements. The resulting vendor locking compared to a solution comprised of multiple small tools is usually not considered.

Is it that it's not considered, or that getting a whole suite of "work nicely together out of the box" tools from a single vendor is looked on as a better option in most cases?

Especially when things go wrong, having one contact point can be quite valuable, instead of having to go through the cat-herding that is getting someone to agree it's their component that's the problem.

Re: Beyond CI/CD: GitLab's DevOps Vision (2017)

#38

Post by mariusmg »

Cthulhu_ wrote: post #19
eadz wrote: post #18

I like gitlab, but a mixed open source / closed source product is always going to be a challenge in terms of hearts and minds. Take this quote from the article. > The other way to look at it is that this is pretty advanced stuff, and frankly, it doesn’t deserve to be, free, open source. So is all the stuff that gitlab builds on, like git, or ruby, or linux, that's all pretty advanced stuff I'd say.

Wow that really is a poorly worded phrase. If someone from GL is reading this, please fix that, it makes you look pretty bad. Just be honest and say you want to charge for advanced features, because there's little money to be made in open source and you're a business after all and want to pay your employees a living wage.

>it makes you look pretty bad.

But they are being honest, they are saying what they are thinking....

Re: Beyond CI/CD: GitLab's DevOps Vision (2017)

#39

Post by leg100 »

oblio wrote: post #2

Is anyone here using the Gitlab CI/CD for anything a bit more complex? I'm especially interested in a pipeline with parallel executions of different kinds of tests, maybe some manual checkpoint in there, a more complex chain with execution on different kinds of hosts or containers.

We're got some pretty complex pipelines, so much so that we've reached quite a few limitations.

You can do all the things you've listed. But you can't build an arbitrarily complex DAG of tasks. Instead, you must order everything into a sequential list of 'stages'. Within a stage you have a list of jobs, which are run in parallel. This might be enough for your needs, but I'd like flexibility to express dependencies for each and every task, Say `t1 -> ((t2,t3),t4 -> t5) -> t6`, etc.

And it's a pain to propagate variables from one task to another, like say setting an environment variable in one task, and getting its value in a downstream task. You end up writing scripts to write them to a file, telling gitlab to make that file an 'artifact', and then reading it downstream.

And then there's the transient errors, all of which are listed on open issues. We're using the hosted gitlab.com, and on some of our CI pipelines, we have a dozen or some stages. The chances of a transient error, .e.g. a runner getting a 503 from gitlab.com, are pretty high.

Re: Beyond CI/CD: GitLab's DevOps Vision (2017)

#40

Post by Vinnl »

eadz wrote: post #18

I like gitlab, but a mixed open source / closed source product is always going to be a challenge in terms of hearts and minds. Take this quote from the article. > The other way to look at it is that this is pretty advanced stuff, and frankly, it doesn’t deserve to be, free, open source. So is all the stuff that gitlab builds on, like git, or ruby, or linux, that's all pretty advanced stuff I'd say.

The formulation is a bit weird. I can't find it right now, but I recall reading in the past that features that are typically only useful when an instance has 100 or more users are the ones that are not made open source. That said, they've open sourced often requested features [0].

[0] https://gitlab.com/gitlab-org/gitlab-ce/issues/34591#note_56...

Post reply on HN