Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Concourse CI

concourse.ci

Post reply on HN
Print view View original

71–80 of 158 posts

Re: Concourse CI

#71

Post by gabrielr »

mugsie wrote: post #64

I know everyone hates on Jenkins these days, but (at least for the workloads I am building) Concourse feels like a toy. My issues: - everything has to be in a docker container (not all things can run in garden) - Just trying to find logs is a pain, and getting the UI to show a full log output is basically impossible - each install of concourse may need an entirely different version of fly - There was no way of retrig…

> Just trying to find logs is a pain, and getting the UI to show a full log output is basically impossible

Have you tried `fly watch`? It will stream logs from in-flight builds, or show the full output of completed ones.

Re: Concourse CI

#72

Post by mdelillo »

rjzzleep wrote: post #16

I've been eyeing Concourse and Go.CD over Jenkins for a while. The main criticism I saw on Jenkins and Go.CD vs. Concourse was that Jenkins Pipelines aren't first class and that it's easier to export configuration(in that regard Concourse > Go.CD > Jenkins). On the other hand Jenkins and Go.CD supports extensions, which Concourse touts as a feature. I also want the CI builds to create my base boxes with packer in mul…

> I also want the CI builds to create my base boxes with packer in multiple steps.

This is certainly possible using concourse, I've done it myself on a few teams. We had one job that started with a base ISO and used the virtualbox-iso builder to apply updates and build an OVA. Then a second job would trigger whenever a new OVA was built and used the virtualbox-ovf to apply a different set of provisioners. Since the virtualbox-ovf builder uses an OVA/OVF file as both it's input and output, you can do that as many times as you'd like.

Re: Concourse CI

#73

Post by wlamartin »

fatterypt wrote: post #65
wlamartin wrote: post #61

Earlier quoted context omitted.

You can achieve this by clicking on a resource and hitting toggling the "power button" to off for each of the resources you don't want to include in the build.

While this may even work, it is not practical and it is a disappointing experience for a pipeline centric type of CI such as concourse. In other tools, hitting re-run would simply replay the job, with the same state it was executed in the first run. I would expect concourse to behave similarly, but no.

I guess I'm not clear on why you would be wanting to do this so frequently that the toggle is impractical, though it's likely that I simply haven't come across this use case.

Re: Concourse CI

#74

Post by mugsie »

gabrielr wrote: post #71
mugsie wrote: post #64

I know everyone hates on Jenkins these days, but (at least for the workloads I am building) Concourse feels like a toy. My issues: - everything has to be in a docker container (not all things can run in garden) - Just trying to find logs is a pain, and getting the UI to show a full log output is basically impossible - each install of concourse may need an entirely different version of fly - There was no way of retrig…

> Just trying to find logs is a pain, and getting the UI to show a full log output is basically impossible Have you tried `fly watch`? It will stream logs from in-flight builds, or show the full output of completed ones.

sure - I can use fly. if I got to the page, find the link to download the right version of fly, then remember the archine incantation to login, find a job, and find the logs.

Or - I can go to my Jenkins build page, and click "Show full log".

For me, I find the initial quick searching through logs much quicker in a browser.

And, I have found that on longer builds the web UIs log streaming can actually crash the browser.

Re: Concourse CI

#75

Post by mugsie »

wlamartin wrote: post #67
mugsie wrote: post #64

I know everyone hates on Jenkins these days, but (at least for the workloads I am building) Concourse feels like a toy. My issues: - everything has to be in a docker container (not all things can run in garden) - Just trying to find logs is a pain, and getting the UI to show a full log output is basically impossible - each install of concourse may need an entirely different version of fly - There was no way of retrig…

> not all things can run in garden Can you provide an example of what you'd want to do but you aren't able to?

Booting VMs is kind of the main one.

Re: Concourse CI

#77

Post by joncampbelldev »

mugsie wrote: post #64

I know everyone hates on Jenkins these days, but (at least for the workloads I am building) Concourse feels like a toy. My issues: - everything has to be in a docker container (not all things can run in garden) - Just trying to find logs is a pain, and getting the UI to show a full log output is basically impossible - each install of concourse may need an entirely different version of fly - There was no way of retrig…

For a lightweight pipeline based CI tool (with no docker requirements) you may like buildkite. I use it for almost everything I used to do in jenkins, although I still keep jenkins around as a glorified cron job / easy generic task runner.

(not associated with buildkite, just a very happy customer)

Re: Concourse CI

#78

Post by gabrielr »

mariusmg wrote: post #43
witten wrote: post #14

I took part in evaluating Concourse CI for the needs of my company (30+ devs). While it has amazing CI pipeline capabilities, we ultimately didn't select Concourse because it felt much more like a CI toolbox, requiring some development to put those tools to use. And what we really wanted was more of a turnkey CI product. Perhaps ironically, we ended up doing some development around the edges of the CI product we ulti…

>requiring some development to put those tools to use The best CI is the one where you write code to build/deploy etc. Wrangling with some moronic UI exposed by the CI product to abstract code is simply wrong. Don't do CI/CD like this.

Right. One of my favorite parts of Concourse is that all the config is declarative, in YAML that teams can (and do) check into a public Git repo (with the secrets kept elsewhere).

That lets teams share Concourse task and resource definitions. Many teams (including mine) publish our pipeline definitions too [0]. This enables "tooling" type teams build components that get a lot of re-use by other teams.

disclaimer: I work for Pivotal.

[0] e.g. code.cloudfoundry.org/cf-networking-ci

Re: Concourse CI

#79

Post by perlgeek »

webo wrote: post #11

Mostly off-topic but I've been looking for more than a CI for some quite time, more on the CD side. How are you guys handling some of these cases? Bonus points for hosted options. - Truly a pipeline based stages. No messing around with git branches/tags for each environment release. - Ability to combine multiple builds together. - Build dependencies. Ability to trigger project-B build when project-A build succeeds. (…

We use https://gocd.io/ so I'll comment on this:

> - Truly a pipeline based stages. No messing around with git branches/tags for each environment release.

That's how we use it. We have two or more stages for each environment (repo upload + installation, often tests) that are simply chained.

> - Ability to combine multiple builds together.

You can do fan-in and fan-out between pipelines (you can combine multiple pipelines in a DAG).

> - Build dependencies. Ability to trigger project-B build when project-A build succeeds. (Docker Cloud Builds have something like this.)

As above, with a graph of pipelines. We use this to trigger rebuild of some projects when libraries change.

> - Use the same artifact across multiple build stages.

You can reference and retrieve artifacts from all previous stages, even stages from upstream pipelines. The syntax for referencing takes some time to get used to.

> - An option to promote a build to the next step either manually or automatically.

automatic is default, but you can have manual approval steps, optionally with permissions attached to them.

Plug: I wrote a book that uses GoCD for its examples: https://leanpub.com/deploy

Re: Concourse CI

#80

Post by mugsie »

joncampbelldev wrote: post #77
mugsie wrote: post #64

I know everyone hates on Jenkins these days, but (at least for the workloads I am building) Concourse feels like a toy. My issues: - everything has to be in a docker container (not all things can run in garden) - Just trying to find logs is a pain, and getting the UI to show a full log output is basically impossible - each install of concourse may need an entirely different version of fly - There was no way of retrig…

For a lightweight pipeline based CI tool (with no docker requirements) you may like buildkite. I use it for almost everything I used to do in jenkins, although I still keep jenkins around as a glorified cron job / easy generic task runner. (not associated with buildkite, just a very happy customer)

buildkite looks nice - but on premise is a current requirement, as not all our repos are on the internet. Looks like a cool product though
Post reply on HN