Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern CI is too complex and misdirected

gregoryszorc.com

Post reply on HN
Print view View original

121–130 of 184 posts

Re: Modern CI is too complex and misdirected

#121

Post by zaat »

lenkite wrote: post #82

I so oppressed with YAML chosen as the configuration language for mainstream CI systems. How do people manage to live with this ? I always make mistakes - again and again. And I can never keep anything in my head. It's just not natural. Why couldn't they choose a programming language ? Restrict what can be done by all means, but something that has a proper parser, compiler errors and IDE/editor hinting support would…

VS code with the prettier extension is an IDE with hinting, parser and immediately show when you have (not compiler but) errors. If there is an extension for your CI system try installing it too.

Re: Modern CI is too complex and misdirected

#122

Post by cyberpunk »

ElijahLynn wrote: post #114
bob1029 wrote: post #33

We got tired of using external tools that were not well-aligned with our build/deployment use cases - non-public network environments. GitHub Actions, et. al. cannot touch the target environments that we deploy our software to. Our customers are also extremely wary of anything cloud-based, so we had to find an approach that would work for everyone. As a result, we have incorporated build & deployment logic into our s…

> GitHub Actions, et. al. cannot touch the target environments that we deploy our software to. It can with on-prem self-hosted Runners > https://docs.github.com/en/actions/hosting-your-own-runners/... I just had this same complaint about using Actions and was pointed to this document.

Yeah and that's a total shitshow too. You want to run that in k8s?

    * First off it won't work with a musl image
    * You need to request a new token every 2 hours
    * It doesn't spawn pods per build like gitlab, it's literally a dumb runner, the jobs will execute *IN* the runner container, so no isolation, and you need all the tools under the sun installed in the container (our runner image clocked in at 2gb for a java/node stack)
    * Get prepared for a lot of .net errors on your linux boxes (yes, not exactly a showstopper but.. urgh).
I hated my time with GitHub actions and will not miss it.

Re: Modern CI is too complex and misdirected

#123

Post by stitched2gethr »

lenkite wrote: post #82

I so oppressed with YAML chosen as the configuration language for mainstream CI systems. How do people manage to live with this ? I always make mistakes - again and again. And I can never keep anything in my head. It's just not natural. Why couldn't they choose a programming language ? Restrict what can be done by all means, but something that has a proper parser, compiler errors and IDE/editor hinting support would…

This is what chef did and I enjoyed it, but it seems that's not the way most systems went.

Re: Modern CI is too complex and misdirected

#125

Post by therealx »

brown9-2 wrote: post #103
jayd16 wrote: post #9

This makes no sense to me. Modern build systems have reproducible results based on strict inputs. Modern CI/CD handles tasks that are not strictly reproducible. The continuous aspect also implies its integrated to source control. I guess I don't understand the post if its not just semantic word games based on sufficient use of the word sufficient. Maybe the point is to talk about how great Taskcluster is but the only…

Your CI pipeline builds and tests your project, which is the same thing your build system does, except they are each using different specifications of how to do that. The author argues this is a waste. I think by introducing continuous deployment you are changing the topic from what the author wrote (which strictly referred to CI).

I would think the build system would only build the binaries or other compiled source. The CI pipeline would build and test.

Re: Modern CI is too complex and misdirected

#126

Post by sltkr »

pydry wrote: post #98
reubenmorais wrote: post #97

Earlier quoted context omitted.

That's kind of what Bazel does. Skylark is a Python dialect.

Why is an entirely new dialect necessary? Why couldn't it just have been a python library?

Starlark is an extremely limited dialect of Python. It is intentionally not Turing complete, to prevent unbounded computation.

I think the rationale for making it syntactically similar to Python was: we want to add a macro processing language to our build system, and we want to support common features like integers, strings, arithmetic expressions, if-statements, for-loops, lists, tuples and dictionaries, so why not base our DSL off of Python that has all that stuff, so that people familiar with Python will have an easy time reading and writing it?

Then they implemented a very limited but very fast interpreter that supports just the non-Turing-complete DSL.

Re: Modern CI is too complex and misdirected

#127

Post by swlkr »

pydry wrote: post #11

It's weird that people keep building DSLs or YAML based languages for build systems. It's not a new thing, either - I remember using whoops-we-made-it-turing complete ANT XML many years ago. Build systems inevitably evolve into something turing complete. It makes much more sense to implement build functionality as a library or set of libraries and piggyback off a well designed scripting language.

Pulumi and CDK come to mind, they look very interesting compared to yaml/dsl approaches

Re: Modern CI is too complex and misdirected

#128

Post by theptip »

qbasic_forever wrote: post #112
theptip wrote: post #43

Earlier quoted context omitted.

> Every time I see "nindent" in yaml code, a part of my soul turns to dust. Yup. For this reason it's a real shame to me that Helm won and became the lingua franca of composable/configurable k8s manifests. The one benefit of writing in static YAML instead of dynamic , is that regardless of primary programming language, everyone can contribute; more complex systems like KSonnet start exploding in first-use complexity.

I wouldn't say helm has won, honestly. The kubectl tool integerated Kustomize into it and it's sadly way too underutilized. I think it's just that the first wave of k8s tutorials that everyone has learned from were all written when helm was popular. But now with some years of real use people are less keen on helm. There are tons of other good options for config management and templating--I expect to see it keep chang…

Are there good kustomize bases for things like Redis, Postgres, Mysql, etc? My impression is that most "cloud-native" projects ship raw manifests and Helm charts, or just helm charts. By "won" I just mean in terms of community mind-share, not that they built the best thing. But I could be out of date there.

I do like kustomize, but the programming model is pretty alien, and they only recently added Components to let you template a commonly-used block (say, you have a complex Container spec that you want to stamp onto a number of different Deployments).

Plus last I looked, kustomize was awkward when you actually do need dynamic templating, e.g. "put a dynamic annotation onto this Service to specify the DNS name for this review app". Ended up having to use another program to do templating which felt awkward. Maybe mutators have come along enough since I last looked into this though.

Re: Modern CI is too complex and misdirected

#129

Post by thinkharderdev »

ryan29 wrote: post #109
thinkharderdev wrote: post #102

Earlier quoted context omitted.

It's funny. If you stick around this business long enough you see the same cycles repeated over and over again. When I started in software engineering, builds were done with maven and specified using an XML config. If you had to do anything non-declarative you had to write a plugin (or write a shell script which called separate tasks and had some sort of procedural logic based on the outputs). Then it was gradle (or…

It's so true. I used Ant > Maven > Gradle. The thing that I think is different about modern CI is there's no good, standard way of adding functionality. So it's almost never write a plugin and always hack something together. And none of it's portable between build systems which are (almost) all SaaS, so it's like getting the absolute worst of everything. I'll be absolutely shocked if current CI builds still work in 1…

Haha. I'd be surprised if they work NEXT year.

Re: Modern CI is too complex and misdirected

#130

Post by tmh88j »

cosmodisk wrote: post #116
cogman10 wrote: post #87

Earlier quoted context omitted.

lol... yeah.... I've become "the expert" on so much shit just because of RTFM :D It's amazing how much stuff is spelt out in manuals that nobody bothers to read. The only issue is that so few people RTFM that some manuals are pure garbage to try and glean anything useful. In those cases, usually the best route is often to just read the implementation (though that is tedious).

I have someone in my network, who is very active in PHP scene. Tutorials, tips&tricks, code reviews, you name it. He pretty much abandoned his very popular website and went all in on YouTube. Why? Apparently watching video is so much easier than reading 3000 word article.

>He pretty much abandoned his very popular website and went all in on YouTube. Why? Apparently watching video is so much easier than reading 3000 word article.

Learning from videos is lazy? Content you've learned is only valid if you read it? I'm not sure what exactly you're getting at, but I'm a visual learner and I much prefer (well made) videos over text. There's a visual and audio aspect to it enabling so much more information to be conveyed at once. For example, Dr. Sedgwick's video series on algorithms and data structures has animations as he steps through how algorithms work. He's overlaying his audio while displaying code and animating the "data" as it's processed by the algorithm. I have a physical copy of his book Algorithms but I go back to his videos when I need a refresher. https://www.youtube.com/watch?v=Wme8SDUaBx8

Post reply on HN