Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We built the fastest CI and it failed

earthly.dev

Post reply on HN
Print view View original

211–220 of 301 posts

Re: We built the fastest CI and it failed

#211

Post by Klugistiono »

gabereiser wrote: post #117
AaronM wrote: post #107

Earlier quoted context omitted.

Did you mean argowf instead of argocd? Argocd is pretty simple and only syncs k8 resources.

I mean that node (not js) dashboard to build pipelines and make things look logical instead of writing 4 lines of bash. I'm not a fan of anything that adds complexity. ArgoCD/WF/ALL ARGO... is just a pretty dashboard over cli tooling. ./deploy.sh @ This is the contract you should focus on. Whatever you need to make ./deploy.sh do this, do it.

Have you ever used ArgoCD?

No one EVER should run deploy.sh like how you propose it.

ArgoCD abstracts away access to a k8s cluster (login through dex).

It has strategies of how to deploy k8s resources like through retry, replace (which you would need to write manually in your deploy.sh).

It gives you debug output for your deploy process (yes thats a thing, you do not assume every deploy just works).

ArgoCD also can and should be used to reference secerts from secret store like vault.

And it gives you a history/audit.

Re: We built the fastest CI and it failed

#212

Post by Klugistiono »

shepherdjerred wrote: post #84
bastardoperator wrote: post #54

It failed because the marketing is outright bunk and overly dishonest. If I compile with Jenkins, Actions or Earthly, that compile time is going to be the same under each build system assuming the same build node. Claiming you're 20x faster when CI is firing within seconds is kind of meaningless. Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. CI is all…

> Caching and parallel execution are age old concepts when it comes to CI and every modern build system can do it. It is much harder to set up caching/parallelization with GitHub Actions, Jenkins, Docker, and Make than it is with just Earthly

Most paralization is a junit configuration or parallel jobs. Which is super easy in Github Actions.

Re: We built the fastest CI and it failed

#213

Post by howinteresting »

pharmakom wrote: post #97
alexvitkov wrote: post #15

Earlier quoted context omitted.

Incremental builds are something even enterprise build systems fail at. I've had Visual Studio and MSBuild fail on me multiple times by deciding not to recompile a certain .cpp file, because they got the dependency graph wrong. This results in a running but inconsistent executable that's a bitch to debug. If you're selling me another tool that duplicates this work, but that has even less context on my project than th…

Bazel does this. I have to be honest - MS Build does not scale well. Companies use it because it’s the default.

Yes, but Bazel requires an all-in commitment and also tons of work to fit the rest of the world into it.

Re: We built the fastest CI and it failed

#214

Post by hhh »

gabereiser wrote: post #106
hhh wrote: post #62

Earlier quoted context omitted.

I feel almost certain someone could wrap ArgoCD into a product if they handled the 'patternization' of applicationsets. D2iQ already do it with Flux, but we bailed on D2iQ before we even got to try it.

ArgoCD is almost worthless if your DevOps teams are IaC wielders. Honestly, just a bash script to push a container to the pods is all that’s required for DevOps deployment. A build pipeline should be get artifacts -> put artifacts in container -> push container to registry -> trigger a pull on your pods . If you didn’t have a standard cloud architecture, I can understand the frustration and reasoning behind looking a…

My needs are more 1 application, 150 configurations, 150 clusters.

Re: We built the fastest CI and it failed

#215

Post by rrrix1 »

psYchotic wrote: post #187
gabereiser wrote: post #117

Earlier quoted context omitted.

I mean that node (not js) dashboard to build pipelines and make things look logical instead of writing 4 lines of bash. I'm not a fan of anything that adds complexity. ArgoCD/WF/ALL ARGO... is just a pretty dashboard over cli tooling. ./deploy.sh @ This is the contract you should focus on. Whatever you need to make ./deploy.sh do this, do it.

Isn't ArgoCD more of a GitOps tool? The pretty UI is mostly secondary to its main purpose for me, which is to keep the declarative "truth" in source control, and have ArgoCD be the control loop that keeps the cluster in sync with that truth. Accidentally nuked namespace? No worries, ArgoCD's (or whatever alternative, like flux) got your back!

Exactly this. The critical difference being who deploys and when/how your resources are deployed - a human, or a machine.

Having git be the source of truth of your production environment is a blessing and a curse which is directly related to the maturity of the deployment system.

If you have low confidence in your deployments (as in, you don't deploy very often and don't have full e2e tests & monitoring), GitOps is nightmarishly scary compared to classic, battle tested Ops team CLI scripts.

Re: We built the fastest CI and it failed

#216

Post by chubot »

Terretta wrote: post #37
chubot wrote: post #20

It was honestly a bit hard for me to understand what happened from this write-up. I read it a few times, and the terms are still bit confusing. Seems like there are at least two separate issues - migration of the CI configuration -- from existing CI YAML (github/gitlab) to Earthly (Makefile/Dockerfile hybrid) - migration of the job runners -- from existing CI to a service hosted by Earthly I would have thought the fi…

It's not about migrating the syntax. It's that people's CI over time become some kind of amalgamated model encapsulating how a firm makes every individual piece of software it makes and lands it places, think (ab)using a CI as Airflow (arbitrary automation job runner), and that the migration is first reverse engineering what people used to know, before even starting untangling all of that to express it some different…

OK but my point is that should have been apparent at the step 1 validation.

Seems like they are onto step 2 or 3, and figured out that step 1 wasn't actually validated -- people didn't want to migrate to Earthly files ???

The blog post is not clear about what happened

Re: We built the fastest CI and it failed

#217

Post by bluGill »

cyberpunk wrote: post #146
bluGill wrote: post #28

Earlier quoted context omitted.

Why should senior devs know that? Someone needs to know those differences, but on a large enterprise project you have a team that keeps the CI system working. The rest need to know how to checkin a file and how to check the CI output. They shouldn't care about those details. I expect any senior developer CAN learn the above things. However that doesn't mean they are worth knowing.

Just to be clear, your question is: why should a senior developer -- whose job ultimately involves managing a complex set of text files -- understand basic concepts behind the industry standard tooling for managing ... text files? Complete waste of time, they should be out playing golf with a board member instead, right..

They need as much as a junior to get the job done. Then they should focus on the hard problems many of which are not tooling related.

However if a different team has made a good ci system they don't need to know how it works, just enough to diagnose a build error they didn't see locally. Likewise, someone needs to know the details of git, but there is no need for everyone to know the details of that mess (git is a particularly bad version control, I wish the world had settled on something else - there are several better choices) so long as you can do the basic commits, merges and checkouts. Most senior developers need to know how to add a text file to cmake (or whatever your build system is), but there is a lot more you can do with a build system that only a few need to know.

Senior devs need to be great at the problem domain they are in. That is normally the subsystem they write code in, and good architecture therein. They need to be able to figure out all the tools you list and more, but becoming experts in them all is a waste of time if enough others already are.

Re: We built the fastest CI and it failed

#218

Post by mindwok »

reactordev wrote: post #136
prometheon1 wrote: post #131

Earlier quoted context omitted.

I thought using your own runners for free in Gitlab was already an option before Github Actions even existed

When it was an open source project, yes. You are correct. Ever since GitLab went commercial, it's been behind a paywall. Apparently you can run runners on self-managed CE so I'm going to look into that.

GitLab is still an open source project (the core of it is MIT licensed) and using your own runners has always been an option. Also, free accounts on the SaaS version get free CI/CD minutes as well.

Re: We built the fastest CI and it failed

#219

Post by BoorishBears »

justin_oaks wrote: post #202
BoorishBears wrote: post #160

Earlier quoted context omitted.

As others are pointing out, once you've grown big enough that you can't trust everyone with the keys go the kingdom: enterprise pricing is for you. SSO Tax is a fun meme, but it misses the point: Whether it's 5 people or even 3, you've scaled your business enough to hire people who aren't in the inner circle. That's a fair place to consider you an enterprise.

Not sure what you're talking about. It's not like without SSO everyone in the company has to all use the same password. Services allow people to have separate users with separate privileges.

It sounds like you're unaware of why SSO is considered a security feature at all them, but it's covered right on the site: https://sso.tax/

It's to allow centralized access management. Stuff like firing someone and revoking their access from one platform instantly, instead running around and changing permissions in every tool manually. Or ensuring people in department A can't be invited to some platform for people in department B in order to limit information access.

SSO tax is predicated on the idea that the moment you outgrow the informal arrangements and liberal access, you're really a business. Seems pretty fair?

Re: We built the fastest CI and it failed

#220

Post by nivenhuh »

reactordev wrote: post #136
prometheon1 wrote: post #131

Earlier quoted context omitted.

I thought using your own runners for free in Gitlab was already an option before Github Actions even existed

When it was an open source project, yes. You are correct. Ever since GitLab went commercial, it's been behind a paywall. Apparently you can run runners on self-managed CE so I'm going to look into that.

I run a self-managed CE GitLab with my own runners — works as expected (no payment required).
Post reply on HN