Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jenkins Is Getting Old

itnext.io

Post reply on HN
Print view View original

321–330 of 340 posts

Re: Jenkins Is Getting Old

#321

Post by hardwaresofton »

chupasaurus wrote: post #320
hardwaresofton wrote: post #317

Earlier quoted context omitted.

Multi repo pipelines are not paywalled? I can have one repo trigger jobs in another today, I've written about it a little bit: https://vadosware.io/post/fun-with-gitlab-ci/ Am I misunderstanding what you meant?

Answered to other sibling comment, could elaborate further for you though. In order to pull other pipelines through Gitlab's API, you have to build another CI on top of an existing one. What I mean is that you're making the a pipeline to handle projects, but there's a catch - you have to run another pipeline to set the variables in the One pipeline to rule them all because you can't override variables from the jobs t…

Thanks for clarifying, I see exactly what you mean now. It looks like some Gitlabbers are in this thread now so hopefully they see this as well

Re: Jenkins Is Getting Old

#322

Post by solatic »

falsedan wrote: post #288
solatic wrote: post #104

OP really needs to try Concourse. Same container-based workflow as Drone that is touted as a solution, but more mature, and much more testable than Drone. Concourse really hits his requirements for ops-friendliness and testability. It's easy to upgrade because the web host and CI workers are completely stateless, and the work you do with Concourse is easy to test because the jobs themselves are all completely self-co…

> ops-friendliness certainly I love the idea of concourse as a release engineer but the lack of a nice UI for dev feedback/monitoring makes it a hard sell as a drop-in jenkins replacement

So, there is a resource that will fetch your pull requests so that they can be built. It's not quite as good as per-branch builds, but with GitHub's new draft pull request feature (if you use GitHub), it does the trick for us, but we're also a relatively small dev team.

Either way, it's not a drop-in Jenkins replacement. It really does have a high learning curve because it forces you to wrap your mind and your code to Concourse's model. Probably, a lot of your build and deployment scripts would need to be rewritten. The reason why you would do so is to get the benefits described above - everything (including environments) is version controlled, ops are stateless, running code in the CI environment from a developer machine, etc.

Re: Jenkins Is Getting Old

#323

Post by ay »

stevekemp wrote: post #257
ay wrote: post #239

Earlier quoted context omitted.

I’m experimenting right now with how far I can simplify the abstractions, and writing my own thing in rust. Since my use case is integration with gerrit, I poll the updated changes over ssh, and have the regex-based triggers which cause a “job” launches. Job consists of making a database entry and calling a shell script, then updating the entry upon completion. Since job is just a shell script it can kick off other j…

I think these kind of home-grown systems are pretty hard to "sell" to others. I know that I've written a couple, my general approach was to : * Get triggered by a github (enterprise) webhook. * Work out the project, and clone it into a temporary directory. * Launch a named docker container, bind-mounting the temporary directory to "/project" inside the image. * Once the container exits copy everything from "/output"…

Oh yeah, agree absolutely. I don’t plan on “selling” this toy outside its original intended audience, just the timing was funny :)

Re: Jenkins Is Getting Old

#324

Post by RealSpamapS »

djhaskin987 wrote: post #241
FrenchyJiby wrote: post #18

I sincerely wish I could move away from Jenkins for the reasons stated in TFA (GUI-oriented, slow, hard to backup/config, test-in-production mentality and boundless plugins) but I've never found something that fits the bill. The much-touted repo integrations (travis, circle...) all have an exclusive focus on build-test-deploy CI of single repos. But when you have many similar repos (modules) with similar build steps…

Zuul-ci.org had recently caught my eye, particularly because it fully supports heavy integration testing of multi-repo apps. It doesn't yet have support for bitbucket server though, which is sort of a deal breaker for me.

It's just a simple matter of code. You're not the first to want BitBucket support.

Re: Jenkins Is Getting Old

#325

Post by RealSpamapS »

Anyone who agrees with the OP should give Zuul-ci.org a look.

I run Zuul in production with GitHub and AWS, and it's been really useful for us. It scales to the moon, parallelizes everything, and having all job/pipeline configuration in git means you can test things like CI job changes before they're committed.

Cross-repo pre-merge dependency management means I can build whole speculative futures in PR's before it lands and breaks the build.

Funny story, the Zuul build status badge is just a static image, because Zuul does not allow code to merge if it fails tests: https://zuul-ci.org/docs/zuul/user/badges.html

Full disclosure: I worked on a defunct zuul based service for a little while, and still am a core developer with lots of code in Zuul. A few presentations I've given are here:

http://fewbar.com/the-build-is-never-broken/ http://fewbar.com/zuul-ci-crossing-streams/

Re: Jenkins Is Getting Old

#326

Post by Robin_f »

hardwaresofton wrote: post #316
majkinetor wrote: post #314

Earlier quoted context omitted.

TBH, its easily done in shell via REST API. We use custom pipeline runner from simple Powershell scripts that works even better for us then default style. We set all jobs to manual and then our script triggers them depending on commit message, person, moon phase etc. But really, this should be in core. Its very hard to do multirepository stuff. Its not that easy to do mono repo stuff too - I really need a pipeline wi…

I find it very easy to make multi-repo stuff, I've also wrote about it: https://vadosware.io/post/fun-with-gitlab-ci Also, running pipelines in different projects and whenever you want is accessible from the web interface... It could be cleaner, but IMO adding even more syntax to the YAML file can also be a rabbit hole -- CURL from the script section seems like not a bad middle ground. Am I misunderstanding what is b…

This post seems very focused on projects with just a single developer. The "recognizing and tagging" versions for example completely breaks down when you have multiple people commit and merging things to master and multiple pipelines running that are trying to recognize and tag versions.

Re: Jenkins Is Getting Old

#327

Post by ogtifs »

oblio wrote: post #298
ogtifs wrote: post #284

Anyone looking for a more back-to-basics self-hosted CI might be interested in Laminar[1] - this article reminds me why I was inspired to write it [1] https://laminar.ohwg.net/

It's not distributed, from what I see. I don't see stuff about agents. Oh, it does, via DYI SSH, sort of. Could work, I guess.

Right, the philosophy is more about making it easy to DIY stuff and getting out of your way, rather than endlessly tacking on features and integrations.

Re: Jenkins Is Getting Old

#328

Post by majkinetor »

Robin_f wrote: post #326
hardwaresofton wrote: post #316

Earlier quoted context omitted.

I find it very easy to make multi-repo stuff, I've also wrote about it: https://vadosware.io/post/fun-with-gitlab-ci Also, running pipelines in different projects and whenever you want is accessible from the web interface... It could be cleaner, but IMO adding even more syntax to the YAML file can also be a rabbit hole -- CURL from the script section seems like not a bad middle ground. Am I misunderstanding what is b…

This post seems very focused on projects with just a single developer. The "recognizing and tagging" versions for example completely breaks down when you have multiple people commit and merging things to master and multiple pipelines running that are trying to recognize and tag versions.

This is what interests me a lot.

I have found that there are moments that start to give chaotic behavior in CI that are influenced by the number of people working on mono repository and some other stuff like code maturity: 1 dev, 2-5 devs, 5-10 devs and 10+ devs .....

Something that works great for any number might start to break for other number. Something effective for huge team may be very bad for smaller team. Something that works great in one moment (first several months of the project when thing still get shape 0.X.Y versions) sux when we move to 1.x version and vice versa.

I am not aware of anybody giving more detailed thought about this nor examine what could be most optimal solution for any type of team (if such thing exist) or on what precise moment you start doing certain practices (for example protecting master branch).

Re: Jenkins Is Getting Old

#329

Post by _whiteCaps_ »

jniedrauer wrote: post #148
solatic wrote: post #104

OP really needs to try Concourse. Same container-based workflow as Drone that is touted as a solution, but more mature, and much more testable than Drone. Concourse really hits his requirements for ops-friendliness and testability. It's easy to upgrade because the web host and CI workers are completely stateless, and the work you do with Concourse is easy to test because the jobs themselves are all completely self-co…

The thing that turned me off concourse last time I checked it out is that their documentation assumes (assumed?) you're going to use BOSH. I don't want to have to learn and maintain yet another infrastructure as code tool, just for my build server. I know you can run concourse without it, but all their examples seemed to use it and I didn't want to hit edge cases that they didn't account for. So I gave up before too…

There's a helm chart available for Concourse, I haven't tried it yet though. Definitely agree about BOSH making it harder to get started.

https://github.com/helm/charts/tree/master/stable/concourse

Re: Jenkins Is Getting Old

#330

Post by boleary-gl »

chupasaurus wrote: post #320
hardwaresofton wrote: post #317

Earlier quoted context omitted.

Multi repo pipelines are not paywalled? I can have one repo trigger jobs in another today, I've written about it a little bit: https://vadosware.io/post/fun-with-gitlab-ci/ Am I misunderstanding what you meant?

Answered to other sibling comment, could elaborate further for you though. In order to pull other pipelines through Gitlab's API, you have to build another CI on top of an existing one. What I mean is that you're making the a pipeline to handle projects, but there's a catch - you have to run another pipeline to set the variables in the One pipeline to rule them all because you can't override variables from the jobs t…

GitLab Product Manager for Verify (CI) here.

This is really valuable feedback, thank you for it. It's something we are thinking about and working on fleshing out a vision for. Two epics we have around these ideas include [1] Making CI lovable for monorepos and [2] Making CI lovable for microservices.

Both of these challenge our current assumptions around the project::pipeline relationship and will help make room for improvements around that model and provide flexibility to build more complex models that work for "real world" problems like the ones you've stated. However, I don't believe we have all the answers yet so I would love more feedback on these and the issues attached.

[1] https://gitlab.com/groups/gitlab-org/-/epics/812

[2] https://gitlab.com/groups/gitlab-org/-/epics/813

Post reply on HN