Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern CI is too complex and misdirected

gregoryszorc.com

Post reply on HN
Print view View original

131–140 of 184 posts

Re: Modern CI is too complex and misdirected

#131

Post by [deleted] »

gryn wrote: post #117
stingraycharles wrote: post #31

Earlier quoted context omitted.

I agree with you in principle, but I have learned to accept that this only works for 80% of the functionality. Maybe this works for a simple Diablo or NodeJS project, but in any large production system there is a gray area of “messy shit” you need, and a CI system being able to cater to these problems is a good thing. Dockerizing things is a step in the right direction, at least from the perspective of reproducibilit…

I'm curious what's a Diablo Project ? I've never heard of such technology unless you're speaking of the game with the same name. Did you possibly mean Django ?

[deleted]

Re: Modern CI is too complex and misdirected

#132

Post by jugg1es »

bob1029 wrote: post #33

We got tired of using external tools that were not well-aligned with our build/deployment use cases - non-public network environments. GitHub Actions, et. al. cannot touch the target environments that we deploy our software to. Our customers are also extremely wary of anything cloud-based, so we had to find an approach that would work for everyone. As a result, we have incorporated build & deployment logic into our s…

Forgive me, but it sounds dangerous and insecure to give your software the kind of access that would be required to do what you described. Even with safety measures and auditing in place, I'm not sure if I would feel comfortable doing this.

Re: Modern CI is too complex and misdirected

#133

Post by bob1029 »

jugg1es wrote: post #132
bob1029 wrote: post #33

We got tired of using external tools that were not well-aligned with our build/deployment use cases - non-public network environments. GitHub Actions, et. al. cannot touch the target environments that we deploy our software to. Our customers are also extremely wary of anything cloud-based, so we had to find an approach that would work for everyone. As a result, we have incorporated build & deployment logic into our s…

Forgive me, but it sounds dangerous and insecure to give your software the kind of access that would be required to do what you described. Even with safety measures and auditing in place, I'm not sure if I would feel comfortable doing this.

How is this any less secure than handing the customer a zip file containing arbitrary binary files and asking them to execute them with admin privileges?

Re: Modern CI is too complex and misdirected

#134

Post by brown9-2 »

therealx wrote: post #125
brown9-2 wrote: post #103

Earlier quoted context omitted.

Your CI pipeline builds and tests your project, which is the same thing your build system does, except they are each using different specifications of how to do that. The author argues this is a waste. I think by introducing continuous deployment you are changing the topic from what the author wrote (which strictly referred to CI).

I would think the build system would only build the binaries or other compiled source. The CI pipeline would build and test.

Which build tools don’t also run tests? That seems very common with Java, Scala, Bazel users, npm, etc.

Re: Modern CI is too complex and misdirected

#135

Post by gravypod »

jrockway wrote: post #17

I think that modern CI is actually too simple. They all boil down to "get me a Linux box and run a shell script". You can do anything with that, and there are a million different ways to do everything you could possibly want. But, it's easy to implement, and every feature request can be answered with "oh, well just apt-get install foobarbaz3 and run quuxblob to do that." A "too complex" system, would deeply integrate…

This is something that I love about using Bazel. It allows you to do this. Bazel is aware of application-level concepts: libraries, binaries, and everything that glues this together. It has a simple way to describe a "test" (something that's run who's exit code determines pass/fail) and how to link/build infinitely complex programs. Do you build a game engine and need to take huge multi-gb assets folders and compile them into an efficient format for your game to ship with? You can use a genrule to represent this and now you, your CI, and everyone on your team will always have up-to-date copies of this without needing to worry about "Bob, did you run the repack script again?"

It also provides a very simple contract to your CI runners. Everything has a "target" which is a name that identifies it.

A great talk about some things that are possible: https://youtu.be/muvU1DYrY0w?t=459

At a previous company I got our entire build/test CI (without code coverage) from ~15 minutes to ~30 to ~60 seconds for ~40 _binary and ~50 _tests (~100 to ~500 unit tests).

Re: Modern CI is too complex and misdirected

#136

Post by jugg1es »

I always thought of CI to be a build server that pushes code directly to production automatically. I never liked how CI does away with the concept of a 'release' with a specific version. To me CI is synonymous with SaaS products, specifically products whose job is to serve individual client requests via a front end. I've never understood how CI is supposed to work with different types of software that isn't so transactional, or when the transactions are much longer running than a single HTTP request.

Re: Modern CI is too complex and misdirected

#137

Post by hardwaresofton »

theptip wrote: post #43
gilbetron wrote: post #13

Earlier quoted context omitted.

Joe Beda (k8s/Heptio) made this same point in one of his TGI Kubernetes videos: https://youtu.be/M_rxPPLG8pU?t=2936 I agree 100%. Every time I see "nindent" in yaml code, a part of my soul turns to dust.

> Every time I see "nindent" in yaml code, a part of my soul turns to dust. Yup. For this reason it's a real shame to me that Helm won and became the lingua franca of composable/configurable k8s manifests. The one benefit of writing in static YAML instead of dynamic , is that regardless of primary programming language, everyone can contribute; more complex systems like KSonnet start exploding in first-use complexity.

Some of us use Make + evnsubst[0] (and more recently make + kustomize[1]) in defiance.

I haven't found time to take a look at Helm 3 yet though, it might be worth switching to.

[0]: https://www.vadosware.io/post/using-makefiles-and-envsubst-a...

[1]: https://www.vadosware.io/post/setting-up-mailtrain-on-k8s/#s...

Re: Modern CI is too complex and misdirected

#138

Post by hardwaresofton »

swlkr wrote: post #127
pydry wrote: post #11

It's weird that people keep building DSLs or YAML based languages for build systems. It's not a new thing, either - I remember using whoops-we-made-it-turing complete ANT XML many years ago. Build systems inevitably evolve into something turing complete. It makes much more sense to implement build functionality as a library or set of libraries and piggyback off a well designed scripting language.

Pulumi and CDK come to mind, they look very interesting compared to yaml/dsl approaches

Fully agreed -- Pulumi was in this space correctly and right out of the gate while CDK is a relative newcomer (both in general and inside the walled garden of AWS). AWS has contributed to the bloodshed with CloudFormation for a long time.

Also, don't forget that CDK for Terraform now exists[0] as well.

[0]: https://www.hashicorp.com/blog/cdk-for-terraform-enabling-py...

Re: Modern CI is too complex and misdirected

#139

Post by jugg1es »

bob1029 wrote: post #133
jugg1es wrote: post #132

Earlier quoted context omitted.

Forgive me, but it sounds dangerous and insecure to give your software the kind of access that would be required to do what you described. Even with safety measures and auditing in place, I'm not sure if I would feel comfortable doing this.

How is this any less secure than handing the customer a zip file containing arbitrary binary files and asking them to execute them with admin privileges?

I wouldn't do that either, but it's even less secure than that because the software would have credentials to the source control system. It also means your source control system has to be public.

Re: Modern CI is too complex and misdirected

#140

Post by cosmodisk »

tmh88j wrote: post #130
cosmodisk wrote: post #116

Earlier quoted context omitted.

I have someone in my network, who is very active in PHP scene. Tutorials, tips&tricks, code reviews, you name it. He pretty much abandoned his very popular website and went all in on YouTube. Why? Apparently watching video is so much easier than reading 3000 word article.

>He pretty much abandoned his very popular website and went all in on YouTube. Why? Apparently watching video is so much easier than reading 3000 word article. Learning from videos is lazy? Content you've learned is only valid if you read it? I'm not sure what exactly you're getting at, but I'm a visual learner and I much prefer (well made) videos over text. There's a visual and audio aspect to it enabling so much mo…

There are pros and cons of each media, I'm not arguing against it. What I mean is that there's a general tendency of migration from longer versions( books) to more compact,bite size content(videos). It's one of the reasons why there's countless videos on YouTube on how to create a dictionary in Python,even though the documentation has it covered wide and deep.
Post reply on HN