Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shifting Gears

jenkins.io

Post reply on HN
Print view View original

81–90 of 123 posts

Re: Shifting Gears

#81

Post by jacques_chester »

cfontes wrote: post #75
jacques_chester wrote: post #67

Earlier quoted context omitted.

Concourse's components (ATC, DB, Workers) can run on Kubernetes, but it is still handling the scheduling of containers it creates. Delegating container scheduling to Kubernetes is the next major epic on the Core track for Concourse. As for Argo: I am not particularly in favour of Turing-complete YAML. Disclosure: I really like Concourse. I work for Pivotal, which sponsors Concourse development.

Valid point. Don't fly you end up creating a turing-complete config file as well? I remember it being stored as some kind of Yaml like syntax.

Fly does substitutions, but that's it: no loops, no if-thens. It's almost certain that something can be tickled into being Turing-complete; it's actually devilishly difficult to avoid doing it by accident. My hunch is pipelines are Turing complete, but I haven't gotten around to proving it.

But there's a difference between trying not to introduce it and going out of your way to build a programming language in YAML. An actual, honest-to-god programmable YAML.

Re: Shifting Gears

#82

Post by jacques_chester »

yebyen wrote: post #35
sytse wrote: post #24

Earlier quoted context omitted.

Thanks for using GitLab! I'm glad to hear you found it easy to set up. I just wrote an article in response to the OP https://about.gitlab.com/2018/09/03/how-gitlab-ci-compares-w... We're working on making GitLab more performant. It is mostly fixes to our code, the parts where ruby is a problem are already rewritten in Go. GitLab self-hosted should be fast if it has enough memory, so make sure you check on its memory…

> The current legacy version of Jenkins needs to be restarted once a day by an administrator Is this true? Do you have a source that says this? We have a Jenkins instance that Kubernetes is configured to scale down from 1 replica to 0 at night, and up to 1 again in the morning, so if it is true we never would have noticed. (It hasn't always run on this cron cycle, which is why I'm a little incredulous at this claim,…

> Honestly I don't understand this about "making a version of Jenkins that runs well on Kubernetes" – this is the _only way_ I have ever run Jenkins, and I think it runs already extraordinarily well for our purposes.

I think the idea is not that Jenkins runs on Kubernetes, which as you note can already be done. It's rather that Jenkins uses Kubernetes as a replacement for the worker infrastructure.

Re: Shifting Gears

#83

Post by yannoux »

orf wrote: post #19
Rowern wrote: post #17

Earlier quoted context omitted.

I've been working with gitlab CI for the last year. Here are some of my feedbacks: - 6 months ago we seriously considered moving away because it was really unstable (even when running on private runners) but now its a lot smoother - with private runners you can have a very powerful CI without having to manage a master (as Jenkins) for a fraction of the costs (runner with docker-machine on spot instances) - beware tha…

> it will seem a good idea to run gitlab-runner on every laptop of your team to reduce cost. Will it?!

Gitlab runner is really easy to install on Linux. At work, I run Gitlab-CI jobs on my laptop : the main reason was the shared runners (provided by my company) were unstable and full. Our Gitlab instance has now ~20 shared runners (used by dozens of teams) and are a lot more stable. I still use my laptop to avoid waiting forever for the docker images to be downloaded.

Re: Shifting Gears

#84

Post by vorg »

nrclark wrote: post #21

I love Jenkins, and use it professionally. With that said, I'd really like to see better documentation on the Jenkinsfile Pipeline format. I've tried to get started with it a few times, and haven't had tons of success. Stuff like "How do I pull in secrets", and "How do I control a plugin". I appreciate that it's Groovy-based, but that's not particularly helpful information (for a hack like me, at least). The snippet-…

You can't just use the docs at Apache Groovy's website because Jenkins pipeline uses a crippled version of Groovy -- none of the functional collections-based methods work.

Re: Shifting Gears

#85

Post by shoo »

nrclark wrote: post #21

I love Jenkins, and use it professionally. With that said, I'd really like to see better documentation on the Jenkinsfile Pipeline format. I've tried to get started with it a few times, and haven't had tons of success. Stuff like "How do I pull in secrets", and "How do I control a plugin". I appreciate that it's Groovy-based, but that's not particularly helpful information (for a hack like me, at least). The snippet-…

After working in a team that was heavily using Jenkins files & scripted pipelines I started to believe that writing Jenkins scripted pipelines is a bit of an anti pattern, as you end up with lots of build script code that can only run inside of a Jenkins, perhaps coupled to plugins, which hampers your ability to locally develop and test changes.

Perhaps sometimes using Jenkins scripted pipeline is a good idea, but if you've got the choice of implementing something as a Jenkins pipeline script or some other script that isn't coupled to Jenkins, prefer the latter.

Re: Shifting Gears

#86

Post by jacques_chester »

cfontes wrote: post #68
twic wrote: post #32

Earlier quoted context omitted.

First i've heard of Argo CI. Why would i want that rather than Concourse? Which can now be deployed on Kubernetes, it seems: https://github.com/helm/charts/tree/master/stable/concourse

I am no Concourse expert, but I've tried it a couple times. Concourse's interface is just beautiful and functional, the CLI is great as well, but it has it's own ecosystem. Lot's of tools to solve problems that K8s already solves (Jobs, Volumes, resources, scheduling, etc..) it also uses workers which if I remember correctly is just like a scheduled slave which you need to install stuff into. Argo is a different beas…

> Lot's of tools to solve problems that K8s already solves

In fairness, Concourse predates Kubernetes and was invented developed to support a team working on a different container scheduler (Diego, similar vintage to Kubernetes).

Otherwise you're right. The pain of carrying around its own container scheduling, worker management etc has been a notably hefty thorn in the side of Concourse for the last two years or so. It should go away as Concourse gets rebased onto Kubernetes.

> The Syntax, is K8S YAML syntax, the same with a couple more things, very little learning curve if you know K8S YAMLs, Concourse is a very different thing.

As part of my work in and around Knative Build, Topher Bullock and I wrote a little façade Kubernetes controller[0] that lets you send pipelines and see builds from kubectl. Basically it looks like an existing Concourse pipeline, but with the usual k8s boilerplate.

I'm not sure what the final Concourse-on-Kubernetes will look like. kubectl is showing one-size-fits-all discomfort (see, for example, knctl[1]) and I am not sold that CRDs are suitable for every purpose. 2018 is clearly The Year of the CRD, so I figure we're about 18 months away from the trough of disillusionment on those.

[0] https://github.com/jchesterpivotal/knative-build-pipeline-po...

[1] https://github.com/cppforlife/knctl

Re: Shifting Gears

#87

Post by twic »

jacques_chester wrote: post #67
twic wrote: post #32

Earlier quoted context omitted.

First i've heard of Argo CI. Why would i want that rather than Concourse? Which can now be deployed on Kubernetes, it seems: https://github.com/helm/charts/tree/master/stable/concourse

Concourse's components (ATC, DB, Workers) can run on Kubernetes, but it is still handling the scheduling of containers it creates. Delegating container scheduling to Kubernetes is the next major epic on the Core track for Concourse. As for Argo: I am not particularly in favour of Turing-complete YAML. Disclosure: I really like Concourse. I work for Pivotal, which sponsors Concourse development.

Wait, doesn't Pivotal also sponsor Turing-complete YAML development?

Re: Shifting Gears

#88

Post by ak217 »

Having used Hudson/Jenkins for many years, I recently considered setting it up for a new project, and backed away mostly due to the issues Kohsuke describes. We ended up choosing GitLab instead.

GitLab has been pulling ahead in features and usability, compared to other things I've tried. Right now, different projects I'm involved with use a combination of GitLab Enterprise, Travis, Circle, and Google Cloud Build. Of those, GitLab accommodates the heaviest and most sophisticated workloads, without having to go through too much trouble to set up, maintain, and instruct developers how to use it (certainly less trouble than Jenkins). I highly recommend taking a critical look at all of these services, to see which best fits your needs.

Re: Shifting Gears

#89

Post by auxym »

shoo wrote: post #85
nrclark wrote: post #21

I love Jenkins, and use it professionally. With that said, I'd really like to see better documentation on the Jenkinsfile Pipeline format. I've tried to get started with it a few times, and haven't had tons of success. Stuff like "How do I pull in secrets", and "How do I control a plugin". I appreciate that it's Groovy-based, but that's not particularly helpful information (for a hack like me, at least). The snippet-…

After working in a team that was heavily using Jenkins files & scripted pipelines I started to believe that writing Jenkins scripted pipelines is a bit of an anti pattern, as you end up with lots of build script code that can only run inside of a Jenkins, perhaps coupled to plugins, which hampers your ability to locally develop and test changes. Perhaps sometimes using Jenkins scripted pipeline is a good idea, but if…

I work with Jenkins day in, day out.

Doing anything build-script related in Jenkins, whether Pipeline or freestyle jobs, is definitely an anti-pattern. All build-related scripts should definitely be in standalone scripts / build tool config files (make or whatnot), for reasons you describe.

Jenkins should be there to handle the "side effects", as I view them. In our case that's stuff like integration with git PRs (posting results of linting, building, unit tests), sending emails when new builds are available, integration with JIRA (we automate some workflows), publishing artifacts to an internal server, etc.

Conversely, putting any of those side-effects or stateful steps inside build scripts is a bad idea, and it leads to not being able to run build scripts locally without worry of messing up a JIRA workflow or spamming people with build emails. Thus, they should be stored only in Jenkins.

These are all mistakes of my predecessors that I am still living with to this day.

Re: Shifting Gears

#90

Post by michaelneale »

auxym wrote: post #89
shoo wrote: post #85

Earlier quoted context omitted.

After working in a team that was heavily using Jenkins files & scripted pipelines I started to believe that writing Jenkins scripted pipelines is a bit of an anti pattern, as you end up with lots of build script code that can only run inside of a Jenkins, perhaps coupled to plugins, which hampers your ability to locally develop and test changes. Perhaps sometimes using Jenkins scripted pipeline is a good idea, but if…

I work with Jenkins day in, day out. Doing anything build-script related in Jenkins, whether Pipeline or freestyle jobs, is definitely an anti-pattern. All build-related scripts should definitely be in standalone scripts / build tool config files (make or whatnot), for reasons you describe. Jenkins should be there to handle the "side effects", as I view them. In our case that's stuff like integration with git PRs (po…

I think thats a great rule of thumb. Declarative pipeline came after the script was "invented", which is slightly unfortunate, had it came before it would have encouraged the practices you describe (declarative is just for orchestration), and script would have been mainly an escape hatch (I think many people get the idea now though).
Post reply on HN