Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We built the fastest CI and it failed

earthly.dev

Post reply on HN
Print view View original

241–250 of 301 posts

Re: We built the fastest CI and it failed

#241

Post by oftenwrong »

bastardoperator wrote: post #166
marcinzm wrote: post #140

Earlier quoted context omitted.

That doesn't help with non-deterministic failures and I've also yet to see a true "write once, run anywhere" system ever. It may be 99% "write once, run anywhere" but there's always that 1% edge case.

Exactly. This just reminds me of "works on my computer". Building locally is more advantageous for developing CI workflow, but I want to be as close to prod as possible and doing that on snowflake developer workstations is an exercise in futility.

Earthly makes use of BuildKit, which essentially executes the build steps in containers. It provides more isolation from the CI runner / dev workstation. Instead of having developers manage their own build tools, Earthly makes it easy to have the build definition manage them.

Re: We built the fastest CI and it failed

#242

Post by oftenwrong »

nvm0n2 wrote: post #125
danudey wrote: post #61

Earlier quoted context omitted.

That's step one, 'how to write a basic Earthfile that just does the thing', so they don't delve into the details of caching and how it works, but look at the whole block and not just that one line; it operates like a Dockerfile: build: COPY build.gradle ./ COPY src src RUN gradle build RUN gradle install SAVE ARTIFACT build/install/java-example/bin /bin SAVE ARTIFACT build/install/java-example/lib /lib The COPY phase…

A CI system like TeamCity will give you that sort of thing for free though, without needing an Earthfile. It just dedupes builds at the same commit, and keeps checkout directories between builds on the agents, so if the build tool has a cache then it's automatically reused. Works great. Given that it's hard to understand what exactly this is for? Build systems normally try to avoid redundant work for local purposes a…

One of the chief benefits of Earthly is builds that are always clean and predictable. This is especially important on a shared build server.

Re: We built the fastest CI and it failed

#243

Post by gabereiser »

jcrawfordor wrote: post #226
reactordev wrote: post #136

Earlier quoted context omitted.

When it was an open source project, yes. You are correct. Ever since GitLab went commercial, it's been behind a paywall. Apparently you can run runners on self-managed CE so I'm going to look into that.

(GitLab employee but familiar with this because I use it personally as well) You can always connect your own runners to GitLab, whether SaaS or self-hosted, and whether free/CE or premium/ultimate/EE. GitLab SaaS also comes with free use of the central runners up to a quota. I'm not sure about the history here but as far as I know this has always been the case.

Circa 2013? I remember vividly implementing gitlab at a certain cable company I worked at because they were too cheap for GitHub. I remember sometime in 2012 some merging of worlds of GitLab CI and GitLab.com, it was around that timeframe until 2015 or so when the unofficial runner because official and CI and Repos were merged? You work there, ask the oldest grey beard if this was the case. I could very well be mistaken and have false memory.

Re: We built the fastest CI and it failed

#245

Post by gabereiser »

rhaway84773 wrote: post #171
gabereiser wrote: post #144

Earlier quoted context omitted.

This. AzDO will die. It will be replaced entirely by Github. It originally was only designed as a way to get people onto Azure using their already-existing MS partnership to give it away for free. The only purpose, drive adoption of Azure Cloud. In runners, resources, stickiness, and preventing abandoning ship.

Azure Devops existed well before GitHub was owned by MS under Visual Studio Online moniker. It was intended to be a SAAS version of the on Prem offerings MS has provided since before GitHub even existed.

I know it’s history and I know it’s future.

Re: We built the fastest CI and it failed

#246

Post by gabereiser »

Klugistiono wrote: post #211
gabereiser wrote: post #117

Earlier quoted context omitted.

I mean that node (not js) dashboard to build pipelines and make things look logical instead of writing 4 lines of bash. I'm not a fan of anything that adds complexity. ArgoCD/WF/ALL ARGO... is just a pretty dashboard over cli tooling. ./deploy.sh @ This is the contract you should focus on. Whatever you need to make ./deploy.sh do this, do it.

Have you ever used ArgoCD? No one EVER should run deploy.sh like how you propose it. ArgoCD abstracts away access to a k8s cluster (login through dex). It has strategies of how to deploy k8s resources like through retry, replace (which you would need to write manually in your deploy.sh). It gives you debug output for your deploy process (yes thats a thing, you do not assume every deploy just works). ArgoCD also can a…

I used it at a Fortune 500. I’ve used it at a Fortune 100. I’ve used it in a small startup. I stand by my statement. It solves problems self inflicted due to crappy cloud architecture and mutable environments. Running a simple deploy script means your DevOps pipelines and cloud are mature enough to handle rollbacks, failures, blue/green deployments, and any migrations applied after a data snapshot (whatever that looks like in your architecture).

Re: We built the fastest CI and it failed

#247

Post by gabereiser »

psYchotic wrote: post #187
gabereiser wrote: post #117

Earlier quoted context omitted.

I mean that node (not js) dashboard to build pipelines and make things look logical instead of writing 4 lines of bash. I'm not a fan of anything that adds complexity. ArgoCD/WF/ALL ARGO... is just a pretty dashboard over cli tooling. ./deploy.sh @ This is the contract you should focus on. Whatever you need to make ./deploy.sh do this, do it.

Isn't ArgoCD more of a GitOps tool? The pretty UI is mostly secondary to its main purpose for me, which is to keep the declarative "truth" in source control, and have ArgoCD be the control loop that keeps the cluster in sync with that truth. Accidentally nuked namespace? No worries, ArgoCD's (or whatever alternative, like flux) got your back!

I don’t accidentally do anything in production. Sorry. Nor do my teams or any developers who have access to my clouds. These kinds of failures don’t happen in my world. You will not have a mutable environment outside of data storage. Period. Nuke all you want, it will repair and redeploy itself.

Re: We built the fastest CI and it failed

#248

Post by dnfjficiv »

tazjin wrote: post #175
JustLurking2022 wrote: post #173

Earlier quoted context omitted.

Seriously, is there an email distro or something to notify Nix users when there's a new post to go troll? It's not suitable for 90%+ of users, which seems a far cry from superior.

It's just very noticeable when half of what the industry seems to spend its time on is solved problems. You and all the others have to learn why touching the hot stove is a bad idea the same way most children do. It's frustrating to observe. It's fine to criticise Nix for its various flaws, but any suggestion of an "alternative" that has learned nothing from it is just a collective waste of time.

We get it, nix is a better mousetrap.

Why are people ignoring you though? Because if it really was the promised land we'd all be there already. Must be something important y'all are missing about the good old fashioned hot stoves.

Re: We built the fastest CI and it failed

#249

Post by Too »

shepherdjerred wrote: post #200
rewmie wrote: post #179

Earlier quoted context omitted.

> It is much harder to set up caching/parallelization with GitHub Actions, Jenkins, Docker, and Make than it is with just Earthly I never used Earthly but I have to call nonsense on this take. Caching and parallelization is a solved problem in GitLab, GitHub, CircleCI, etc. It's so trivial that it's pipelines 101. Caching and parallelization might be easier in Earthly but any improvement over "trivial" is something n…

How do you usually do caching for complex, multi-language projects in this case? How do you guarantee _only_ changed targets are rebuilt when dependencies, like a language version or external library, change? With Earthly you just... write your Earthfile. You get everything for free. Caching and parallelization _also_ works locally, so I see that speedup in development.

Dependency graph is a feature of the build system with better understanding of the language import rules than the CI system can have (or whatever terminology you want to use). Good ones can hermetically, concurrently and deterministic build only the files that were changed.

Earthly is just a thin wrapper on top of docker multi stage builds[1]. Such docker layers are one-dimensional, meaning if one of the steps are calling “make”, that step will have to rebuild that entire layer, not just the one c++ file that changed. There are solutions to this with --mount,type=cache but they take away the reproducibility and and are equally possible with plain dockerfiles.

[1] tbh I don’t know if this is the case, but that’s how others explain it. This is core problem with Earthly. They don’t say how it works, just that you will magically get 20x faster. Without understanding how, I don’t know how to take that seriously and I can only assume it’s snake oil.

I mean, there is a need for a better Dockerfile syntax. But then market it as such! Not as magic 20x faster builds.

Post reply on HN