Live data from Hacker News

We built the fastest CI and it failed

earthly.dev

11–20 of 301 posts

Re: We built the fastest CI and it failed

#11
We work in an industry where senior developers don't know the difference between git and Github. The same seniors don't know the difference between Github actions, Travis CI and have never heard of Jenkins. Hardly anyone cares.

Are you going to sell this to a startup? I doubt it. Are you going to sell it to a smb or enterprise company? Maybe.

Is there a market to make cash there? Sure. Is it what I would want to make and try selling? Hell no.

Re: We built the fastest CI and it failed

#13
post #8

What does fast CI even mean? CI is an overgrown shell script running your build and telling you when it fails. In general build tooling has gotten so slow that the cost of whatever CI runner relative to it should be nil. If you want fast CI you need fast tsc, clang, rustc, etc... not a faster program that calls exec on them. A bit more on topic, if you're selling CI and your business fails it's because people you're…

Perusing this article quickly, it means a CI that is automatically handling things like caching build artifacts so that you don't need to recompile your entire repository every single commit. It's not about a faster program to call exec; it's about a program that knows it need not even call exec.

I looked at Earthly just a couple of weeks back. It’s a tool that makes perfect sense - especially when I was deep in a makefile wishing the caching was as automated as docker.

In the end I decided that I couldn’t face the complexity of dealing with someone else’s hosted tooling. Luckily in this case.

Re: We built the fastest CI and it failed

#14

We work in an industry where senior developers don't know the difference between git and Github. The same seniors don't know the difference between Github actions, Travis CI and have never heard of Jenkins. Hardly anyone cares. Are you going to sell this to a startup? I doubt it. Are you going to sell it to a smb or enterprise company? Maybe. Is there a market to make cash there? Sure. Is it what I would want to make…

Enterprise is moving to bazel or similar, often with build caches, and using partial builds on checkin based on what’s changed.

Re: We built the fastest CI and it failed

#15
post #8

What does fast CI even mean? CI is an overgrown shell script running your build and telling you when it fails. In general build tooling has gotten so slow that the cost of whatever CI runner relative to it should be nil. If you want fast CI you need fast tsc, clang, rustc, etc... not a faster program that calls exec on them. A bit more on topic, if you're selling CI and your business fails it's because people you're…

Perusing this article quickly, it means a CI that is automatically handling things like caching build artifacts so that you don't need to recompile your entire repository every single commit. It's not about a faster program to call exec; it's about a program that knows it need not even call exec.

Incremental builds are something even enterprise build systems fail at.

I've had Visual Studio and MSBuild fail on me multiple times by deciding not to recompile a certain .cpp file, because they got the dependency graph wrong. This results in a running but inconsistent executable that's a bitch to debug.

If you're selling me another tool that duplicates this work, but that has even less context on my project than the build system, I'll pass.

Re: We built the fastest CI and it failed

#16
Judging by the description, that project seems to be another DAG-planning meta-program with input-based memoization. This is the superior approach to fight complexity but, unfortunately, most engineers just don't understand the problem and the solution.

Re: We built the fastest CI and it failed

#17

> Why pay both the CI vendor and us Exactly. Complexity , not speed, is the problem with builds. It makes perfect sense that "another layer of complexity that wraps up all the other layers" was tough to sell.

In this particular domain, performance problems are tightly coupled with complexity.

Re: We built the fastest CI and it failed

#19
post #8

What does fast CI even mean? CI is an overgrown shell script running your build and telling you when it fails. In general build tooling has gotten so slow that the cost of whatever CI runner relative to it should be nil. If you want fast CI you need fast tsc, clang, rustc, etc... not a faster program that calls exec on them. A bit more on topic, if you're selling CI and your business fails it's because people you're…

Perusing this article quickly, it means a CI that is automatically handling things like caching build artifacts so that you don't need to recompile your entire repository every single commit. It's not about a faster program to call exec; it's about a program that knows it need not even call exec.

except not really.. their tutorial [0] includes no-compile languages like python and JS, with only non-trivial compilation being and Java... and how do they do it?

    RUN gradle build
At least in this case, Earthly has no insight into how repository is organized, and it _will_ recompile the entire repository with every single commit.

So what's the real value of it? this basically seems like a better "docker build" alternative, giving nicer inter-layer caching. Maybe will save some time on dependencies if they take some time to install and devs were too lazy to set up own workers and used ephemeral runners instead.

[0] https://docs.earthly.dev/basics/part-1-a-simple-earthfile

Re: We built the fastest CI and it failed

#20
It was honestly a bit hard for me to understand what happened from this write-up. I read it a few times, and the terms are still bit confusing. Seems like there are at least two separate issues

- migration of the CI configuration -- from existing CI YAML (github/gitlab) to Earthly (Makefile/Dockerfile hybrid)

- migration of the job runners -- from existing CI to a service hosted by Earthly

I would have thought the first thing is the hard thing. Changing the language could take months or even years.

But I don't understand what the blog post was saying. I thought they validated that part? People were able to switch?

But then it wasn't validated at the end? The customers were faced with 2 migrations, not 1?

But now what's happening? You're still keeping the Earthly syntax but abandoning the CI? Isn't that the hard part to migrate to? I'm still confused.

Post reply on HN