Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We built the fastest CI and it failed

earthly.dev

Post reply on HN
Print view View original

151–160 of 301 posts

Re: We built the fastest CI and it failed

#151

Post by imiric »

justin_oaks wrote: post #145
gabereiser wrote: post #114

Earlier quoted context omitted.

plugins, hardware, hosting, support, there's a number of ways to offer a service for an open source project. Open Core models come to mind. Another is not having enterprise SSO as part of the core. Or not having clustering. Something that would make a VP think twice about adopting your open source solution vs using you as a vendor.

I know these are examples, so I don't want to pick on you too much, but I have one nitpick: Reserving SSO for enterprise customers is awful. Please don't do it. See https://sso.tax/ for details but I'll quote this from it "SSO is a core security requirement for any company with more than five employees"

If a company requires SSO, they'll surely have a budget for paying for the products and services they use. The majority of OSS users won't have this requirement, and likely won't have the budget to pay for commercial features. So it doesn't make sense to give away a service that has a maintenance cost to everyone, as long as it's fairly priced.

Re: We built the fastest CI and it failed

#152

Post by WorldMaker »

hotnfresh wrote: post #148
WorldMaker wrote: post #137

Earlier quoted context omitted.

AzDO is "decent", depending on your needs, but it's in an unfortunate zombie state where Microsoft is supporting it just enough to keep it stable and keep certain enterprises happy and consistently insisting that AzDO has a roadmap and is still beloved, but it is very clear that all of the actual resources are going to the GitHub side of the house in 2023. (One of the most recent signs of this crazy zombie state that…

Ugh, they’re busy bringing bad project management to GitHub, in a quest to make it suck as much as all the others. The old issues + tags + milestones was perfect. Now it’s the same needlessly-heavy thing as all the rest, with their reimagined “projects” thingy. But then again I’ve not once seen a PM embrace any version of GitHub’s project management, with the only explanation forthcoming being “it’s confusing for non…

It does seem a case where Microsoft dogfooding GitHub may be actively making it worse. They've done several BUILD and GitHub Universe videos "How Microsoft Uses GitHub to Build [X]" and their PMs certainly need all those complex features to track stuff and those videos are the closest to training materials on how to use some of it, and it is definitely a lot going on.

It definitely seems one of those sorts of "every system PMs touch is doomed to become unmanageable complexity" laws and GitHub "Projects" does seem to be exponentially closer to being "Jira 2: Electric Boogaloo" every day.

Re: We built the fastest CI and it failed

#153

Post by 0xbadcafebee »

There is no real business advantage or value add to switching CI/CD systems. The only people that do it no longer have anyone working for them that knows how the existing CI/CD system works, so the new person just replaces it with something else. They're all basically the same crap. Run a job, evaluate its output, maybe run another job, send a notification.

Whenever you run into a problem with a CI/CD system, it's because the system is "opinionated" and "tightly integrated", but of course lacks some obscure and subtle capability that you need to solve a problem. The fact that you are using a specific CI/CD system is what dooms you to eventually need another one to solve the problem that you current one cannot solve.

It's ridiculous that here we are in 2023 and nobody has completed an open source project that replaces general use CI & CD (for modern best practices & use cases; Jenkins is a curse from the pit of hell). Both CI & CD are just loops of jobs and logic that then decides what next job to run. What those jobs do are all different, but so what? Make an open source library of jobs and logic.

(for what it's worth, Drone.io is the best system I've ever used, and WoodpeckerCI is the open source equivalent, but it doesn't have a large community so work is very slow)

Re: We built the fastest CI and it failed

#154

Post by pxc »

I looked briefly at Earthly a while ago for work because I have to write CI integrations for a whole bunch of CI platforms (GitLab, Azure DevOps, Jenkins, and maybe GitHub Actions). I ended up going with a very similar product (i.e., another nice BuildKit frontend with integrations into a bunch of CI systems), Dagger, because the DSL it used for defining pipelines at the time seemed better to me. I ended up deeply regretting this decision, as the Dagger devs have de facto abandoned that language entirely in favor of churning out SDKs for the most popular programming languages— all imperative, all Turing-complete, and all, imo, bad fits for the space. So now I'm back in hell, integrating with all of these systems by hand, hesitant to trust this kind of tooling to a startup again.

To me this is still the most appealing use case of something like Earthly. Push-and-see-what-happens is still the norm on almost all CI systems, and it's an abominable workflow. If you are at a large organization and have to support teams that use a wide variety of CI/CD platforms, something like Earthly could potentially save quite a lot of pain. But the appeal is precisely in supporting existing CI platforms, rather than in adding another one.

Re: We built the fastest CI and it failed

#155

Post by fydorm »

unbalancedevh wrote: post #134

It really bugs me when authors use acronyms or abbreviations without expanding what they mean. It tells the reader, "I didn't write this article for you, so don't bother trying to understand it."

Which terms are you referring to? If you aren't familiar with "CI/CD", then this article really isn't for you, as you haven't run into the problems this startup is trying to solve.

Re: We built the fastest CI and it failed

#156

Post by tazjin »

dpeckett wrote: post #86
reactordev wrote: post #56

This is a good write up of why you shouldn’t give away the house when you open source things. The issue was really this: Earthly being open-source, Earthy Satellite users were already seeing the benefit from 95% of Earthly CI . I’m a huge fan of open source, however, if your business model includes an open source model - you need a differentiator. Beyond blazingly fast(tm). You need a reason for people to offer up th…

I'm not sure feature withholding has traditionally worked out well in the developer space. The churn rate for tooling is so high that if Earthly were to intentionally cripple their product I'm sure the majority of developers would just move on to other inferior, but free, options (eg. taskfile). Maybe those that remain could be converted successfully but it's quite a gamble. A great turn around story in recent times…

In this particular case, developers could also move on to a superior and completely open alternative (Nix), so making even a semi-closed alternative with much fewer people contributing etc. was already a hard sell.

Re: We built the fastest CI and it failed

#157

Post by hotnfresh »

prpl wrote: post #14
consoomer wrote: post #11

We work in an industry where senior developers don't know the difference between git and Github. The same seniors don't know the difference between Github actions, Travis CI and have never heard of Jenkins. Hardly anyone cares. Are you going to sell this to a startup? I doubt it. Are you going to sell it to a smb or enterprise company? Maybe. Is there a market to make cash there? Sure. Is it what I would want to make…

Enterprise is moving to bazel or similar, often with build caches, and using partial builds on checkin based on what’s changed.

Enterprises with strong, opinionated, tech org leadership, maybe.

The rest are shuffling around between solutions that haven’t been state-of-the-art in 15 years or more, and usually purchased or bundled from some giant vendor or another, as usual.

They’re ahead of the curve if they have any standard widely-implemented CI solution that’s fully in use and not one of three (or more) in various stages of being adopted, maintained, and phased out, in fact.

Re: We built the fastest CI and it failed

#158

Post by gabereiser »

justin_oaks wrote: post #145
gabereiser wrote: post #114

Earlier quoted context omitted.

plugins, hardware, hosting, support, there's a number of ways to offer a service for an open source project. Open Core models come to mind. Another is not having enterprise SSO as part of the core. Or not having clustering. Something that would make a VP think twice about adopting your open source solution vs using you as a vendor.

I know these are examples, so I don't want to pick on you too much, but I have one nitpick: Reserving SSO for enterprise customers is awful. Please don't do it. See https://sso.tax/ for details but I'll quote this from it "SSO is a core security requirement for any company with more than five employees"

If you have more than 5 employees, I’m sending you a sales rep to sell SSO to you.

Re: We built the fastest CI and it failed

#159

Post by theamk »

danudey wrote: post #61
theamk wrote: post #19

Earlier quoted context omitted.

except not really.. their tutorial [0] includes no-compile languages like python and JS, with only non-trivial compilation being and Java... and how do they do it? RUN gradle build At least in this case, Earthly has no insight into how repository is organized, and it _will_ recompile the entire repository with every single commit. So what's the real value of it? this basically seems like a better "docker build" alter…

That's step one, 'how to write a basic Earthfile that just does the thing', so they don't delve into the details of caching and how it works, but look at the whole block and not just that one line; it operates like a Dockerfile: build: COPY build.gradle ./ COPY src src RUN gradle build RUN gradle install SAVE ARTIFACT build/install/java-example/bin /bin SAVE ARTIFACT build/install/java-example/lib /lib The COPY phase…

sure, but in a java project, how likely you are to have a commit with no changes to src/? or for a go project, how likely is it that you want to duplicate you entire dependency tree in a clunky, non-parallel dependency description language, vs just doing "COPY * .; RUN go build" and letting golang figure it all out and build in parallel using all the cores?

Don't get me wrong, there is definitely value in caching dependencies, and I can see Earthly approach help if you have, say, a Python app, with 10 minutes of dependency install time, 0 build time, and 10 seconds of tests - you'll see huge improvement. But you'll also see almost the same improvement if you use Dockerfile on a persistent host instead, and as a bonus you won't have to learn new system.

One place Earthly might shine are monorepos with multiple independent projects that even use multiple build systems... but I am not sure how many of those exist _and_ don't have some other caching solution already.

Re: We built the fastest CI and it failed

#160

Post by BoorishBears »

justin_oaks wrote: post #145
gabereiser wrote: post #114

Earlier quoted context omitted.

plugins, hardware, hosting, support, there's a number of ways to offer a service for an open source project. Open Core models come to mind. Another is not having enterprise SSO as part of the core. Or not having clustering. Something that would make a VP think twice about adopting your open source solution vs using you as a vendor.

I know these are examples, so I don't want to pick on you too much, but I have one nitpick: Reserving SSO for enterprise customers is awful. Please don't do it. See https://sso.tax/ for details but I'll quote this from it "SSO is a core security requirement for any company with more than five employees"

As others are pointing out, once you've grown big enough that you can't trust everyone with the keys go the kingdom: enterprise pricing is for you.

SSO Tax is a fun meme, but it misses the point: Whether it's 5 people or even 3, you've scaled your business enough to hire people who aren't in the inner circle.

That's a fair place to consider you an enterprise.

Post reply on HN