Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern CI is too complex and misdirected

gregoryszorc.com

Post reply on HN
Print view View original

141–150 of 184 posts

Re: Modern CI is too complex and misdirected

#141

Post by tmh88j »

cosmodisk wrote: post #140
tmh88j wrote: post #130

Earlier quoted context omitted.

>He pretty much abandoned his very popular website and went all in on YouTube. Why? Apparently watching video is so much easier than reading 3000 word article. Learning from videos is lazy? Content you've learned is only valid if you read it? I'm not sure what exactly you're getting at, but I'm a visual learner and I much prefer (well made) videos over text. There's a visual and audio aspect to it enabling so much mo…

There are pros and cons of each media, I'm not arguing against it. What I mean is that there's a general tendency of migration from longer versions( books) to more compact,bite size content(videos). It's one of the reasons why there's countless videos on YouTube on how to create a dictionary in Python,even though the documentation has it covered wide and deep.

I agree that something like a Python dictionary is probably unnecessary to make a standalone video for, but simple concepts like that also wouldn't be a 3000 word article unless you're looking for literally every nuance and weird quirk the language has to offer.

Re: Modern CI is too complex and misdirected

#142

Post by cratermoon »

jacques_chester wrote: post #38
cratermoon wrote: post #30

He doesn't mention Concourse, but let me say the complexity there is beast.

I've heard Concourse team members describe it by analogy to make, FWIW. One thing Concourse does well that I haven't seen replicated is strictly separating what is stateful and what isn't. That makes it possible to understand the history of a given resource without needing a hundred different specialised instruments.

Yes, if make threw away all the context and everything you had done prior to a target completing, and only gave the outputs of the target to the next target.

Re: Modern CI is too complex and misdirected

#143

Post by lame88 »

I had to integrate Azure Pipelines and wanted to shoot myself in the face. The idea that you are simply configuring a pipeline yaml is just one big lie; it's code, in the world's shittiest programming language using YML syntax - code that you have no local runtime for, so you have to submit to the cloud like a set of punch cards to see that 10 minutes later it didn't work and to try again. Pipelines are code, pure and simple. The sooner we stop pretending it isn't, the better off we'll be.

Re: Modern CI is too complex and misdirected

#144

Post by 0xbadcafebee »

lenkite wrote: post #82

I so oppressed with YAML chosen as the configuration language for mainstream CI systems. How do people manage to live with this ? I always make mistakes - again and again. And I can never keep anything in my head. It's just not natural. Why couldn't they choose a programming language ? Restrict what can be done by all means, but something that has a proper parser, compiler errors and IDE/editor hinting support would…

YAML is not a language, it's a data format. Why does nobody in the entire tech industry know the difference? I didn't even go to school and I figured it out.

Most software today that uses YAML for a configuration file is taking a data format (YAML) applying a very shitty parser to create a data structure, and then feeding that data structure to a function, which then determines what other functions to call. There's no grammar, no semantics, no lexer, no operators, and no types, save those inherent to the data format it was encoded in (YAML). Sometimes they'll look like they include expressions, but really they're just function arguments.

Re: Modern CI is too complex and misdirected

#145

Post by bob1029 »

jugg1es wrote: post #139
bob1029 wrote: post #133

Earlier quoted context omitted.

How is this any less secure than handing the customer a zip file containing arbitrary binary files and asking them to execute them with admin privileges?

I wouldn't do that either, but it's even less secure than that because the software would have credentials to the source control system. It also means your source control system has to be public.

Client environments have tokenized access to source control, which is private. Builds are pegged to specific commit hashes and triggered via an entirely separate authenticated portal, so there is no chance that someone pushes malicious code and it automatically gets built.

There is a certain mutual degree of trust with the environments we are operating in. We do not worry about the customer gaining access to our source code. Much like the customer doesnt worry too much about the mountains of their PII we are churning through on a regular basis.

Re: Modern CI is too complex and misdirected

#146

Post by throwaway894345 »

sly010 wrote: post #85
throwaway894345 wrote: post #79

Earlier quoted context omitted.

> Re: CI vs build system, I guess the difference is that build systems focus on artifacts, while CI systems also focus on side effects. That said, there are bazel packages to push docker images, so it's certainly a very blurry line. I think the CI and build system have basically the same goals, but they're approaching the problem from different directions, or perhaps it's more accurate to say that "CI" is more impera…

Out of curiosity, what's hard about bazel? From my experience the main issue is interoperability with third party build systems. i.e. using a cmake library that was not manually bazeled by someone.

It’s been a few years since I tried to use it, but at the time the documentation was sparse and misleading. I couldn’t tell the right way to write my own rules and there were several concepts which were muddled. Some of the docs suggested I could extend it by implementing my own rules or macros and other docs suggested I had to delve into the extension interface which was a mess of convoluted Java (the kind of OOP that OOP advocates swear is Not True OOP). It certainly seemed like Bazel supports certain kinds of projects roughly aligned to languages, and if you needed to do anything interesting like codegen the answer was not at all obvious. The worst was that the docs all said that Bazel supported Python 3, but there were half a dozen different flags pertaining to Python 3, and after trying every permutation and asking for help, I discovered that the docs were simply wrong and Python 3 support had never actually been implemented due to various issues. It seems you need deep familiarity with Bazel in order to use it in even a basic capacity.

Re: Modern CI is too complex and misdirected

#147

Post by cosmodisk »

tmh88j wrote: post #141
cosmodisk wrote: post #140

Earlier quoted context omitted.

There are pros and cons of each media, I'm not arguing against it. What I mean is that there's a general tendency of migration from longer versions( books) to more compact,bite size content(videos). It's one of the reasons why there's countless videos on YouTube on how to create a dictionary in Python,even though the documentation has it covered wide and deep.

I agree that something like a Python dictionary is probably unnecessary to make a standalone video for, but simple concepts like that also wouldn't be a 3000 word article unless you're looking for literally every nuance and weird quirk the language has to offer.

like an explanation of what 'this' is in JS:)

https://www.amazon.co.uk/You-Dont-Know-JS-Prototypes

Re: Modern CI is too complex and misdirected

#148

Post by vladaionescu »

The vision of this article is a similar vision to earthly.dev, a company I have founded to pursue the issues presented here.

We have built the build system part and are working on completing the vision with the CI part.

We use buildkit underneath as a constraint solver and our workflows are heavily DAG based.

Hundreds of CI pipelines run Earthly today.

I don't fully agree with all the assumptions in the article, including with the fact that the TAM is limited here. CI has been growing at 19% CAGR and also I think there are possibilities for expanding into other areas once you are a platform.

Re: Modern CI is too complex and misdirected

#149

Post by tmh88j »

cosmodisk wrote: post #147
tmh88j wrote: post #141

Earlier quoted context omitted.

I agree that something like a Python dictionary is probably unnecessary to make a standalone video for, but simple concepts like that also wouldn't be a 3000 word article unless you're looking for literally every nuance and weird quirk the language has to offer.

like an explanation of what 'this' is in JS:) https://www.amazon.co.uk/You-Dont-Know-JS-Prototypes

Haha, I'm genuinely intrigued to see the contents of that book. It could certainly be copy/paste from the Mozilla docs, but who knows! I was pleasantly surprised by this video about the JS event loop, although in fairness it is a lecture at a conference rather than a made for youtube video. Regardless, the length of the video made me question how necessary it was but I ended up watching the whole thing and enjoyed it. https://www.youtube.com/watch?v=8aGhZQkoFbQ

Re: Modern CI is too complex and misdirected

#150

Post by bencalam »

simonw wrote: post #5

It genuinely hadn't crossed my mind that a CI system and a build system were different things - maybe because I usually work in dynamic rather than compiled languages? I've used Jenkins, Circle CI, GitLab and GitHub Actions and I've always considered them to be a "remote code execution in response to triggers relating to my coding workflow" systems, which I think covers both build and CI.

Same boat. I am surprised that I had to to scroll so far down to see this comment.
Post reply on HN