Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern CI is too complex and misdirected

gregoryszorc.com

Post reply on HN
Print view View original

51–60 of 184 posts

Re: Modern CI is too complex and misdirected

#51

Post by zomglings »

jdfellow wrote: post #41

I've been wishing for one of my small projects (3 developers) for some kind of "proof of tests" tool that would allow a developer to run tests locally, and add some sort of token to the commit message assuring that they pass. I could honestly do without a ton of the remote-execution-as-a-service in my current GitLab CI setup, and be happy to run my deployment automation scripts on my own machine if I could have some…

You are right. Small teams absolutely do not need to execute code remotely, especially if the cost is having an always on job server.

My team writes test output to our knowledge base:

    bugout trap --title "$REPO_NAME tests: $(date -u +%Y%m%d-%H%M)" --tags $REPO_NAME,test,zomglings,$(git rev-parse HEAD) -- ./test.sh
This runs test.sh and reports stdout and stderr to our team knowledge base with tags that we can use to find information later on.

For example, to find all failed tests for a given repo, we would perform a search query that looked like this: "# #test !#exit:0".

The knowledge base (and the link to the knowledge base entry) serve as proof of tests.

We also use this to keep track of production database migrations.

Re: Modern CI is too complex and misdirected

#52

Post by tremon »

lillecarl wrote: post #23
pydry wrote: post #11

It's weird that people keep building DSLs or YAML based languages for build systems. It's not a new thing, either - I remember using whoops-we-made-it-turing complete ANT XML many years ago. Build systems inevitably evolve into something turing complete. It makes much more sense to implement build functionality as a library or set of libraries and piggyback off a well designed scripting language.

What is a well defined "scripting" language? Lua, Python, Ruby? I do agree it'd be nice with a more general purpose language and a lib like you say, but should this lib be implemented in rust/c so that people can easily integrate it into their own language? Many unknowns but great idea.

I'd say it's not about the capabilities of the language, but the scope of the environment. You need a language to orchestrate your builds and tests (which usually means command execution, variable interpolation, conditional statements and looping constructs), and you need a language to interact with your build system (fetching code, storing and fetching build artifacts, metadata administration).

Lua would be a good candidate for the latter, but its standard library is minimal on purpose, and that means a lot of the functionality would have to be provided by the build system. Interaction with the shell from Python is needlessly cumbersome (especially capturing stdout/stderr), so of those options my preference would be Ruby. Heck, even standard shell with a system-specific binary to call back to the build system would work.

Re: Modern CI is too complex and misdirected

#53

Post by chriswarbo »

jdfellow wrote: post #41

I've been wishing for one of my small projects (3 developers) for some kind of "proof of tests" tool that would allow a developer to run tests locally, and add some sort of token to the commit message assuring that they pass. I could honestly do without a ton of the remote-execution-as-a-service in my current GitLab CI setup, and be happy to run my deployment automation scripts on my own machine if I could have some…

Git pre-commit hooks can run your tests, but that's easy to skip.

I don't know about a "proof of test" token. Checking such a token would presumably require some computation involving the repo contents; but we already have such a thing, it's called 'running the test suite'. A token could contain information about branches taken, seeds for any random number generators, etc. but we usually want test suites to be deterministic (hence not requiring any token). We could use such a token in property-based tests, as a seed for the random number generator; but it would be easier to just use one fixed seed (or, we could use the parent's commit ID).

Re: Modern CI is too complex and misdirected

#54

Post by alexcroox »

That's why I love Buddy (buddy.works), you build your CI pipelines with a UI, all the config and logic is all easily configurable without having to know the magic config key/value combination. Need to add a secrets file or private key; just add it to the "filesystem" and it'l be available during the run, no awkwardly base64ing contents into an environment string. Unfortunately I have to use github actions/CircleCi for my iOS deployments still, but I read MacOS container support is coming soon.

Re: Modern CI is too complex and misdirected

#55

Post by epage »

Anything in CS can be generalized to its purest, most theoretical forms. The question is how usable is it and how much work does it take to get anything done.

See

- https://danstroot.com/2018/10/03/hammer-factories/

- https://web.archive.org/web/20120427101911/http://jacksonfis...

Bazel, for example, is tailored to the needs of reproducible builds and meets its audience where it is at, on the command line. People want fast iteration time and only occasionally need "everything" ran.

Github Actions is tailored for completeness and meets is audience where its at, the PR workflow (generally, a web UI). The web UI is also needed for visualizing the complexity of completeness.

I never find myself reproducing my build in my CI but do find I have a similar shape of needs in my CI but in a different way.

Some things more tailored to CI that wouldn't fit within the design of something like Bazel include

- Tracking differences in coverage, performance, binary bloat, and other "quality" metrics between a target branch and HEAD

- Post relevant CI feedback directly on the PR

Re: Modern CI is too complex and misdirected

#56

Post by dkarl »

noir_lord wrote: post #24
jrockway wrote: post #17

I think that modern CI is actually too simple. They all boil down to "get me a Linux box and run a shell script". You can do anything with that, and there are a million different ways to do everything you could possibly want. But, it's easy to implement, and every feature request can be answered with "oh, well just apt-get install foobarbaz3 and run quuxblob to do that." A "too complex" system, would deeply integrate…

Pretty much. Docker in my experience is the same way, people see docker as the new hotness then treat it like a Linux box with a shell script (though at least with the benefit you can shoot it in the head). One of the other teams had an issue with reproducibility on something they where doing so I suggested that they use a multistage build in docker and export the result out as an artefact they could deploy, they loo…

> treat it like a Linux box with a shell script (though at least with the benefit you can shoot it in the head)

To be fair, that by itself is a game-changer, even if doesn't take full advantage of Docker.

Re: Modern CI is too complex and misdirected

#57

Post by tremon »

twoslide wrote: post #34

Am I too old fashioned in thinking it’s good to define an acronym the first time it’s used? I think many well educated readers wouldn’t know CI

If you're in software development and haven't heard about Continuous Integration, you're definitely old-fashioned. Although fashion moves fast and breaks things here.

Re: Modern CI is too complex and misdirected

#58

Post by rubiquity »

jacques_chester wrote: post #37
Smaug123 wrote: post #3

"Build Systems à la Carte" is not so much ringing a bell as shattering it with the force of its dong. https://www.microsoft.com/en-us/research/uploads/prod/2018/0... To expand, the OP ends with an "ideal world" that sounds to me an awful lot like someone's put the full expressive power of Build Systems à la Carte into a programmable platform, accessible by API.

Nitpick: I think you meant to write "gong".

Call me crazy, but I don't think they did...

Re: Modern CI is too complex and misdirected

#59

Post by tauntz »

Znafon wrote: post #27
tauntz wrote: post #26

I built a CI service many moons ago (was https://greenhouseci.com back then) on the premise that _most_ companies don't actually require a hugely complicated setup and don't actually need all the flexibility / complexity that most CI systems have. After talking to tens and tens of companies with apparent complex CI requirements, I still stand by that assertion. When drilling down into _why_ they say they need all tha…

Is this still working? The certificate is expired.

I parted ways with my child many years ago. The product itself is still around and has gone through 2 rebrands and has evolved a lot. It's now home under https://codemagic.io

Re: Modern CI is too complex and misdirected

#60

Post by daniellarusso »

noir_lord wrote: post #24
jrockway wrote: post #17

I think that modern CI is actually too simple. They all boil down to "get me a Linux box and run a shell script". You can do anything with that, and there are a million different ways to do everything you could possibly want. But, it's easy to implement, and every feature request can be answered with "oh, well just apt-get install foobarbaz3 and run quuxblob to do that." A "too complex" system, would deeply integrate…

Pretty much. Docker in my experience is the same way, people see docker as the new hotness then treat it like a Linux box with a shell script (though at least with the benefit you can shoot it in the head). One of the other teams had an issue with reproducibility on something they where doing so I suggested that they use a multistage build in docker and export the result out as an artefact they could deploy, they loo…

I am very similar. I read through the documentation the first time to get a ‘lay of the land’, so I can deep-dive into the various sections as I require.
Post reply on HN