Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We built the fastest CI and it failed

earthly.dev

Post reply on HN
Print view View original

41–50 of 301 posts

Re: We built the fastest CI and it failed

#41

Post by aftbit »

Terretta wrote: post #37
chubot wrote: post #20

It was honestly a bit hard for me to understand what happened from this write-up. I read it a few times, and the terms are still bit confusing. Seems like there are at least two separate issues - migration of the CI configuration -- from existing CI YAML (github/gitlab) to Earthly (Makefile/Dockerfile hybrid) - migration of the job runners -- from existing CI to a service hosted by Earthly I would have thought the fi…

It's not about migrating the syntax. It's that people's CI over time become some kind of amalgamated model encapsulating how a firm makes every individual piece of software it makes and lands it places, think (ab)using a CI as Airflow (arbitrary automation job runner), and that the migration is first reverse engineering what people used to know, before even starting untangling all of that to express it some different…

I think the key is that nobody "wants" to spend any time at all on CI or their build system. It is one of those necessary evils that you have to do to have functioning software. Once you do it, even if it's a slow buggy pile of hacks, you will just ignore it until the pain gets very very bad.

This was the same problem that caused our team to abandon CircleCI. They kept wanting us to rewrite our CI configs, touting various benefits that we just didn't care about. All we wanted was for our janky broken builds to keep working 90% of the time, like they did before Circle tried migrating this or that.

Re: We built the fastest CI and it failed

#42

Post by xwowsersx »

linuxdude314 wrote: post #40

Anyone else find it strange how the author uses CI as a noun? I've never heard anyone refer to a CI platform/system this way before and it sort of strikes me as a red flag. Also the idea of running CI on your laptop seems questionable at best, downright irresponsible and insecure at worst. Sure faster builds are nice, but by the time you're running your code through CI/CD its not usually after adding a few lines of f…

> Anyone else find it strange how the author uses CI as a noun?

Do you mean using the term "CI" with no other modifiers/qualifiers?

> Also the idea of running CI on your laptop seems questionable at best, downright irresponsible and insecure at worst.

Didn't you also use CI as a noun here? And isn't it a noun?

> Sure faster builds are nice, but by the time you're running your code through CI/CD its not usually after adding a few lines of for a feature, it's when the completed feature ready to merge or some sort of integration test is needed.

I hear you, but not necessarily. You might have a test suite that runs in CI and after you open your PR, you continuing pushing to it and it's nice to have a fast pipeline.

Re: We built the fastest CI and it failed

#43

Post by ajmurmann »

SkyPuncher wrote: post #34

Unless CI is _really_ painful, it's just not that painful. I spend a day cursing about my build when I spin up a new project, then I forget about it for months. Specs run fast enough - especially locally where I only run failing or known-modified specs.

It very much depends on what you are building and testing. I work on a distributed database and while our build takes like a minute, the tests take hours and we run them on clusters we start on-demand. Getting our CI faster, has been a constant effort for years that quickly pays for itself.

Re: We built the fastest CI and it failed

#44

Post by [deleted] »

mrweasel wrote: post #24
hardwaregeek wrote: post #21

At least for me, whenever I have to set up CI or touch CI, it's incredibly painful. I usually end up having to push a bunch of commits to debug stuff, and there's a lot of configuration hell. I don't know if Earthly solves this, perhaps it does. But if I were considering moving onto a new CI, I'd immediately think about having to redo this painful process. And that's at the team level. At the org level, switching CI…

CI systems are important to me, but my requirement is pretty much: Can it run a bash script. Every CI I've ever used has degraded into a bash script runner. It starts out well enough, you try to use all the build in features but at some point you fallback to running a shells script and then slowly more and more move to the script, because it's easier. Someone should just build a bash CI, something that can manage sec…

[deleted]

Re: We built the fastest CI and it failed

#45

Post by aftbit »

linuxdude314 wrote: post #40

Anyone else find it strange how the author uses CI as a noun? I've never heard anyone refer to a CI platform/system this way before and it sort of strikes me as a red flag. Also the idea of running CI on your laptop seems questionable at best, downright irresponsible and insecure at worst. Sure faster builds are nice, but by the time you're running your code through CI/CD its not usually after adding a few lines of f…

I think the idea is to run the CI on your laptop for debugging and building the CI process itself. This dramatically shortens the build-test cycle (for the CI, not your product). Instead of "edit, git push, wait for runners, download logs, stare confusedly, repeat", you have "edit, run locally, watch logs, stare confusedly, repeat". Plus you can get real access to the actual runners, so maybe you can do things like strace, gdb, etc to see what is going wrong.

Re: We built the fastest CI and it failed

#46

Post by jordemort »

I'm glad that they're just shutting down the service and not closing up shop entirely. I really like the tool (to the point that I regularly stop by their careers page); the syntax for an Earthfile is a very sensible and incremental evolution of Dockerfile syntax, and it makes a whole bunch of things easy that were either previously impossible or very awkward with a Dockerfile alone.

I remember when Docker introduced BuildKit and buildx, they were trying to push Dockerfiles as a general-purpose build system (i.e. not just for containers but also for producing file artifacts and the like) - Earthly takes that idea and really delivers on it.

Re: We built the fastest CI and it failed

#47

Post by wpietri »

chubot wrote: post #20

It was honestly a bit hard for me to understand what happened from this write-up. I read it a few times, and the terms are still bit confusing. Seems like there are at least two separate issues - migration of the CI configuration -- from existing CI YAML (github/gitlab) to Earthly (Makefile/Dockerfile hybrid) - migration of the job runners -- from existing CI to a service hosted by Earthly I would have thought the fi…

I'm glad it wasn't just me. I'm interested in both sides of this, the product development side and the dev tools side. But with all that somewhat confusing prose, I ended up in the neighborhood of: "I'm happy for u tho. Or sorry that happened."

Was the heart of this that they assumed that radically faster builds was the killer feature, but they hadn't actually validated that assumption before building it? Or was it more they hadn't really segmented their users, so they hadn't realized that the high-dollar customers had different needs? Or that they just gave away the thing that was the real-problem solver for paying customers?

And given that the somewhat chaotic article was written by the CEO, I have to wonder to what extent the chaos was just a lack of editing versus there being a lot of actual chaos in the company throughout the process. Long ago Steve Blank wrote an article called "Founders and dysfunctional families" [1] where he talked about how a lot of founders are really good at managing chaos because they grew up in it, something that certainly describes me. He added that something that can make the difference between success and failure is the extent to which founders can also handle non-chaos. Because the ones who can't tend to "throw organizational hand grenades into their own companies" to get things back to the level of chaos that they're good at. That observation has given me pause for thought many times in the years since.

[1] https://steveblank.com/2009/05/18/founders-and-dysfunctional...

Re: We built the fastest CI and it failed

#48

Post by kerblang »

mrweasel wrote: post #24
hardwaregeek wrote: post #21

At least for me, whenever I have to set up CI or touch CI, it's incredibly painful. I usually end up having to push a bunch of commits to debug stuff, and there's a lot of configuration hell. I don't know if Earthly solves this, perhaps it does. But if I were considering moving onto a new CI, I'd immediately think about having to redo this painful process. And that's at the team level. At the org level, switching CI…

CI systems are important to me, but my requirement is pretty much: Can it run a bash script. Every CI I've ever used has degraded into a bash script runner. It starts out well enough, you try to use all the build in features but at some point you fallback to running a shells script and then slowly more and more move to the script, because it's easier. Someone should just build a bash CI, something that can manage sec…

Once you have hooks into source control for commit-based callbacks into CI, everything else need only be shell. Like, if you build a triggers-shell-scripts-off-source-repo-hooks web service, does putting the results in a pretty web server really matter for anyone besides executives who want dashboards dashboards dashboards? I'm already sending script output to text files and our chat system via curl to boring web service APIs.

Re: We built the fastest CI and it failed

#49

Post by nbittich »

A quick look at the documentation and I knew I'll bother using it. We have Jenkins, Drone, woodpecker, GitHub action. Targeting devs feels like it's the worst market to get in, specially after years of VC baked web3/GPT/ terminal/ ide / programming language 300000x faster than python with no tradeoffs offers.

Re: We built the fastest CI and it failed

#50

Post by marcosdumay »

trumbitta2 wrote: post #18

Stopped at "Imagine you live in a world where no part of the build has to repeat unless the changes actually impacted it." because I'm already in that world by using Nx and Nx Cloud.

I am still convinced the author was trying to describe Make.
Post reply on HN