Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern CI is too complex and misdirected (2021)

gregoryszorc.com

Post reply on HN
Print view View original

161–170 of 207 posts

Re: Modern CI is too complex and misdirected (2021)

#161

Post by levlaz »

ajb wrote: post #19
jph wrote: post #2

You're 100% right IMHO about the convergence of powerful CI pipelines and full build systems. I'm very curious what you'll think if you try Dagger, which is my tool of choice for programming the convergence of CI and build systems. (Not affiliated, just a happy customer) https://dagger.io/

So, it sounded interesting but they have bet too hard on the "developer marketing" playbook of "just give the minimum amount of explanation to get people to try the stuff". For example, there is a quick start, so I skip that and click on "core concepts". That just redirects to quick start. There's no obvious reference or background theory. If I was going to trust something like this I want to know the underlying theo…

Hello, Dagger employee here.

Thanks so much for taking a look and sharing your feedback! We've heard this feedback in the past and are working on a big docs change that should make this whole experience a lot better for folks that are new to dagger.

https://devel.docs.dagger.io/getting-started/concepts

This should land in the coming weeks.

Re: Modern CI is too complex and misdirected (2021)

#162

Post by pwnna »

arianvanp wrote: post #149
mrbombastic wrote: post #145

Earlier quoted context omitted.

How do you get caching of build steps with this approach? Or do you just not?

Use a modern hermetic build system with remote caching or remote execution. Nix, Bazel, buck, pants. Many options

This is like fighting complexity with even more complexity. Nix and bazel are definitely not close to actually achieving hermetic build at scale. And when they break the complexity increases exponentially to fix.

Re: Modern CI is too complex and misdirected (2021)

#163

Post by [deleted] »

mettamage wrote: post #34

IMO development is too complex and misdirected in general since we cargo cult FAANG. Need AWS, Azure or GCP deployment? Ever thought about putting it on bare metal yourself? If not, why not? Because it's not best practice? Nonsense. The answer with these things is: it depends, and if your app has not that many users, you can get away with it, especially if it's a B2B or internal app. It's also too US centric. The ide…

[deleted]

Re: Modern CI is too complex and misdirected (2021)

#164

Post by keybored »

bluGill wrote: post #160

I disagree. CI and build systems have different responsibilities and so should be different systems. Both are extremely complex because they have to deal with the complex real world. Many people have the idea they can make things simpler. Which is really easy because the basic problems are not that hard. Them someone needs "just one more small feature" which seems easy enough and it is - but the combination of everyo…

> Many people have the idea they can make things simpler. Which is really easy because the basic problems are not that hard. Them someone needs "just one more small feature" which seems easy enough and it is - but the combination of everyone's small feature is complex.

This is becoming the standard refrain for all software.

Re: Modern CI is too complex and misdirected (2021)

#165

Post by IshKebab »

dakiol wrote: post #29
IshKebab wrote: post #7

Yeah I think this is totally true. The trouble is there are loads of build systems and loads of platforms that want to provide CI with different features and capabilities. It's difficult to connect them. One workaround that I have briefly played with but haven't tried in anger: Gitlab lets you dynamically create its `.gitlab-ci.yaml` file: https://docs.gitlab.com/ci/pipelines/downstream_pipelines/#d... So you can hav…

If there’s something worse than a gitlab-ci.yaml file that is a dynamically-generated gitlab-ci.yaml file.

Why? By dynamically generating it you can skip a lot of the nonsense (any kind of conditional rules, optional jobs, etc.).

Re: Modern CI is too complex and misdirected (2021)

#166

Post by bluGill »

keybored wrote: post #164
bluGill wrote: post #160

I disagree. CI and build systems have different responsibilities and so should be different systems. Both are extremely complex because they have to deal with the complex real world. Many people have the idea they can make things simpler. Which is really easy because the basic problems are not that hard. Them someone needs "just one more small feature" which seems easy enough and it is - but the combination of everyo…

> Many people have the idea they can make things simpler. Which is really easy because the basic problems are not that hard. Them someone needs "just one more small feature" which seems easy enough and it is - but the combination of everyone's small feature is complex. This is becoming the standard refrain for all software.

One other difference: CI and build systems are treated as "side projects" and so none of these efforts have thought into the full system. Generally most software has architects, and often it is a 2nd system that after a lot of effort has finally been made to work.

Re: Modern CI is too complex and misdirected (2021)

#167

Post by bob1029 »

arunix wrote: post #142
bob1029 wrote: post #32

I've been able to effectively skip the entire CI/CD conversation by preferring modern .NET and SQLite. I recently spent a day trying to get a GH Actions build going but got frustrated and just wrote my own console app to do it. Polling git, tracking a commit hash and running dotnet build is not rocket science. Putting this agent on the actual deployment target skips about 3 boss fights.

Is there something about .NET that makes this easier?

Self-contained deployments help a lot.

Re: Modern CI is too complex and misdirected (2021)

#168

Post by bigstrat2003 »

ohdeargodno wrote: post #25
donperignon wrote: post #16

2025 and Jenkins still the way to go

The fact that maintaining any Jenkins instance makes you want to shoot yourself and yet it's the least worst option is an indictment of the whole CI universe. I have never seen a system with documentation as awful as Jenkins, with plugins as broken as Jenkins, with behaviors as broken as Jenkins. Groovy is a cancer, and the pipelines are half assed, unfinished and incompatible with most things.

I have zero problems maintaining Jenkins, and have done so at a couple of different jobs in the past. Minimize how many plugins you use and it works great. We use just a handful: configuration as code, credential storage, kubernetes agent support, pipelines, and job DSL (plus their dependencies of course). Everything is easy to manage because it's just config files in a repo, and things just work for us (with only very rare exceptions).

It would probably be more constructive if you elaborated what your issues specifically were. For example, what have you found pipelines to be incompatible with? I've literally never seen anything they don't work with, so I can't really agree with your assessment without specifics. Similarly, I have zero problem with Groovy. If it's just not to your taste then fine, taste is subjective, but I can't see any substantive reason to call it "a cancer".

Re: Modern CI is too complex and misdirected (2021)

#169

Post by merb »

rcxdude wrote: post #155
merb wrote: post #141

Earlier quoted context omitted.

many ppl also understimate how complex it is to satisfy uptime requirements, how to scale out local infrastructure when storage > 10/50/100tb (yeah a single disk can handle that, but what about bit rot, raid stuff, etc) is involved. it gets worse when you need more servers because your ocr process of course needs cpu x so on a beefiy machine you can handle maybe 50 high page documents. but how do you talk to other ma…

>also humans costs way more money than cloud stuff. I the cloud stuff can be managed in like 1 day per month you dont need a real person, if you have real hardware that day is not enough and you soon need a dedicated person, keeping everything up-to-date, etc. In my experience, I have observed the opposite: companies with on-site infrastructure have been able to manage it in the spare time of a relatively small team…

most of the time the "far more complex setup" is mostly easier than the reimplementation of kubernetes with ansible.

Re: Modern CI is too complex and misdirected (2021)

#170

Post by dakiol »

IshKebab wrote: post #165
dakiol wrote: post #29

Earlier quoted context omitted.

If there’s something worse than a gitlab-ci.yaml file that is a dynamically-generated gitlab-ci.yaml file.

Why? By dynamically generating it you can skip a lot of the nonsense (any kind of conditional rules, optional jobs, etc.).

They are suddenly 10x more difficult to change to suit your needs.
Post reply on HN