Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We built the fastest CI and it failed

earthly.dev

Post reply on HN
Print view View original

81–90 of 301 posts

Re: We built the fastest CI and it failed

#81

Post by bastardoperator »

ravivyas wrote: post #64
bastardoperator wrote: post #54

It failed because the marketing is outright bunk and overly dishonest. If I compile with Jenkins, Actions or Earthly, that compile time is going to be the same under each build system assuming the same build node. Claiming you're 20x faster when CI is firing within seconds is kind of meaningless. Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. CI is all…

Can you expand on the word "feedback"? What kind of feedback are you looking from CI?

It's one thing to know a build failed, it's another thing to know why and how a build failed, where it failed, and what the cause was. If I have to swim through or context shift to a separate build system's output to get that, what is this actually doing for me?

Re: We built the fastest CI and it failed

#82

Post by brabel »

danudey wrote: post #61
theamk wrote: post #19

Earlier quoted context omitted.

except not really.. their tutorial [0] includes no-compile languages like python and JS, with only non-trivial compilation being and Java... and how do they do it? RUN gradle build At least in this case, Earthly has no insight into how repository is organized, and it _will_ recompile the entire repository with every single commit. So what's the real value of it? this basically seems like a better "docker build" alter…

That's step one, 'how to write a basic Earthfile that just does the thing', so they don't delve into the details of caching and how it works, but look at the whole block and not just that one line; it operates like a Dockerfile: build: COPY build.gradle ./ COPY src src RUN gradle build RUN gradle install SAVE ARTIFACT build/install/java-example/bin /bin SAVE ARTIFACT build/install/java-example/lib /lib The COPY phase…

Gradle can do the caching of tasks itself in a level of granularity no CI can. If you keep your build cache "live" in the CI, Gradle knows to only recompile what has changed, and to only run tests which changed (or tests something which changed). I am not sure if many other build systems do that, but this is something I would absolutely expect from the build system, not CI. Do you not expect caching when building locally??

The whole idea with CI doing this is wrong IMO. All I want CI to do is call a couple of commands after setting up the environment (which would include the Gradle cache or whatever to speed up things - no need for your revolutionary CI build system).

Re: We built the fastest CI and it failed

#83

Post by gabereiser »

jangxx wrote: post #68
reactordev wrote: post #56

This is a good write up of why you shouldn’t give away the house when you open source things. The issue was really this: Earthly being open-source, Earthy Satellite users were already seeing the benefit from 95% of Earthly CI . I’m a huge fan of open source, however, if your business model includes an open source model - you need a differentiator. Beyond blazingly fast(tm). You need a reason for people to offer up th…

You can self-host Gitlab CI runners and use them even in the free community version.

I know, this is what I’ve been doing for years. (edit on GitHub, not GitLab, didn’t know GitLab allowed community CI now)

Re: We built the fastest CI and it failed

#84

Post by shepherdjerred »

bastardoperator wrote: post #54

It failed because the marketing is outright bunk and overly dishonest. If I compile with Jenkins, Actions or Earthly, that compile time is going to be the same under each build system assuming the same build node. Claiming you're 20x faster when CI is firing within seconds is kind of meaningless. Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. CI is all…

> Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it.

It is much harder to set up caching/parallelization with GitHub Actions, Jenkins, Docker, and Make than it is with just Earthly

Re: We built the fastest CI and it failed

#85

Post by Spivak »

jangxx wrote: post #68
reactordev wrote: post #56

This is a good write up of why you shouldn’t give away the house when you open source things. The issue was really this: Earthly being open-source, Earthy Satellite users were already seeing the benefit from 95% of Earthly CI . I’m a huge fan of open source, however, if your business model includes an open source model - you need a differentiator. Beyond blazingly fast(tm). You need a reason for people to offer up th…

You can self-host Gitlab CI runners and use them even in the free community version.

And honestly I can't even imagine ever using the public runners because they're permissionless. It's way easier to have them live in your own network and doling out permissions via tags that get picked up by different runners with different iam roles.

Re: We built the fastest CI and it failed

#86

Post by dpeckett »

reactordev wrote: post #56

This is a good write up of why you shouldn’t give away the house when you open source things. The issue was really this: Earthly being open-source, Earthy Satellite users were already seeing the benefit from 95% of Earthly CI . I’m a huge fan of open source, however, if your business model includes an open source model - you need a differentiator. Beyond blazingly fast(tm). You need a reason for people to offer up th…

I'm not sure feature withholding has traditionally worked out well in the developer space. The churn rate for tooling is so high that if Earthly were to intentionally cripple their product I'm sure the majority of developers would just move on to other inferior, but free, options (eg. taskfile). Maybe those that remain could be converted successfully but it's quite a gamble.

A great turn around story in recent times though is probably Docker. But that required them to abandon strict open source and make controversial license/product changes.

Re: We built the fastest CI and it failed

#87

Post by jangxx »

gabereiser wrote: post #83
jangxx wrote: post #68

Earlier quoted context omitted.

You can self-host Gitlab CI runners and use them even in the free community version.

I know, this is what I’ve been doing for years. ( edit on GitHub, not GitLab, didn’t know GitLab allowed community CI now)

Then I guess I misunderstood "GitLab restricts CI/CD to paying customers"?

Re: We built the fastest CI and it failed

#88

Post by shepherdjerred »

I've used Earthly for a year and absolutely love it.

It allows you to have two dependencies on the host system for any project -- just Docker and Earthly.

I've used it to build cross-compilers, system software, web apps, my resume, school projects.

The speed up is real. You can definitely get caching and parallelization and containizeration with other build systems, but Earthly's value is that it comes for free with a familiar syntax.

Re: We built the fastest CI and it failed

#89

Post by maxloh »

pharmakom wrote: post #77

The issue is trust. I dislike trusting GitHub / GitLab with my code but I already do, so I choose them for CI. I can already get fast builds using Bazel (or maybe Buck 2) so why do I need another tool / provider?

GitLab is open sourced. You can self-host GitLab (and their CI solution) yourself.

Re: We built the fastest CI and it failed

#90

Post by notatoad »

aftbit wrote: post #41
Terretta wrote: post #37

Earlier quoted context omitted.

It's not about migrating the syntax. It's that people's CI over time become some kind of amalgamated model encapsulating how a firm makes every individual piece of software it makes and lands it places, think (ab)using a CI as Airflow (arbitrary automation job runner), and that the migration is first reverse engineering what people used to know, before even starting untangling all of that to express it some different…

I think the key is that nobody "wants" to spend any time at all on CI or their build system. It is one of those necessary evils that you have to do to have functioning software. Once you do it, even if it's a slow buggy pile of hacks, you will just ignore it until the pain gets very very bad. This was the same problem that caused our team to abandon CircleCI. They kept wanting us to rewrite our CI configs, touting va…

>I think the key is that nobody "wants" to spend any time at all on CI or their build system.

or if they do, they're hosting their own.

if you're selling a hosted CI platform, your sales pitch is "pay us $$ instead of paying your staff $$$ to do devops". if your pitch is "pay us $$ and then also spend $$$ on staff hours to manage our service", it's very unlikely the CI is doing anything useful enough that the staff couldn't just run their own thing in-house.

it doesn't take much configuration to be more difficult than an EC2 instance running a script on a webhook from your version control.

Post reply on HN