Live data from Hacker News

We built the fastest CI and it failed

earthly.dev

281–290 of 301 posts

Re: We built the fastest CI and it failed

#281
post #175

Earlier quoted context omitted.

It's just very noticeable when half of what the industry seems to spend its time on is solved problems. You and all the others have to learn why touching the hot stove is a bad idea the same way most children do. It's frustrating to observe. It's fine to criticise Nix for its various flaws, but any suggestion of an "alternative" that has learned nothing from it is just a collective waste of time.

We get it, nix is a better mousetrap. Why are people ignoring you though? Because if it really was the promised land we'd all be there already. Must be something important y'all are missing about the good old fashioned hot stoves.

I'm trembling at how much you're ignoring me!

Re: We built the fastest CI and it failed

#282

Earlier quoted context omitted.

Neither GitHub nor slack are open source

Both GitHub and Slack have free tiers. Same principle, different software license.

It isn’t the same principle though. Slack and GitHub can sell data to make a profit etc but the problem with open source is people can just host it themselves meaning the creators don’t even have that option.

Re: We built the fastest CI and it failed

#283
post #175

Earlier quoted context omitted.

It's just very noticeable when half of what the industry seems to spend its time on is solved problems. You and all the others have to learn why touching the hot stove is a bad idea the same way most children do. It's frustrating to observe. It's fine to criticise Nix for its various flaws, but any suggestion of an "alternative" that has learned nothing from it is just a collective waste of time.

> You and all the others have to learn why touching the hot stove is a bad idea the same way most children do. It's frustrating to observe. You say that while your ecosystem still can't get away from cutesy childish terms like "flakes" and "pills". Newsflash: nobody cares. I want "a package" and "a system". I am not impressed. Before you look down on others make your own thing look uber-professional and then uproot a…

I tried to write a serious response to your comment, but as I got further into it and you got further into personal attacks and away from substance, I decided to delete it and I'm instead leaving you with this:

You should get off your high horse. Nobody cares about you having been a "CTO" of some random venture-backed company, it doesn't give you any credentials which you can fling at people to "win" debates online. 80% of the people on this forum have some useless title like that in their CV. You being able to write lots of words about irrelevant superficial topics doesn't mean that you have anything useful to say, even if you've previously been led to believe that these are equivalent.

I don't care where you flock. I publish all my Nix related work for free, I don't have a personal stake. Your decisions are your problem.

Re: We built the fastest CI and it failed

#284
post #283

Earlier quoted context omitted.

> You and all the others have to learn why touching the hot stove is a bad idea the same way most children do. It's frustrating to observe. You say that while your ecosystem still can't get away from cutesy childish terms like "flakes" and "pills". Newsflash: nobody cares. I want "a package" and "a system". I am not impressed. Before you look down on others make your own thing look uber-professional and then uproot a…

I tried to write a serious response to your comment, but as I got further into it and you got further into personal attacks and away from substance, I decided to delete it and I'm instead leaving you with this: You should get off your high horse. Nobody cares about you having been a "CTO" of some random venture-backed company, it doesn't give you any credentials which you can fling at people to "win" debates online.…

Speaking of personal attacks, you chose only one part of my comment where I mentioned how I did some decision making, and ignored everything else.

Well, OK, now I know who is on the high horse so let's drop it here.

Shame that tech zealotry never dies.

Re: We built the fastest CI and it failed

#285

Earlier quoted context omitted.

Have you ever used ArgoCD? No one EVER should run deploy.sh like how you propose it. ArgoCD abstracts away access to a k8s cluster (login through dex). It has strategies of how to deploy k8s resources like through retry, replace (which you would need to write manually in your deploy.sh). It gives you debug output for your deploy process (yes thats a thing, you do not assume every deploy just works). ArgoCD also can a…

I used it at a Fortune 500. I’ve used it at a Fortune 100. I’ve used it in a small startup. I stand by my statement. It solves problems self inflicted due to crappy cloud architecture and mutable environments. Running a simple deploy script means your DevOps pipelines and cloud are mature enough to handle rollbacks, failures, blue/green deployments, and any migrations applied after a data snapshot (whatever that look…

Very weird that you have such an strong opinion of having some self written deploy script over a argocd setup.

ArgoCD is super easy to setup, has a declarative setup and is a great platform tool.

I introduced it to big systems and i really can't imagine how we would struggle if we would deploy those infra components with some bash scripts.

Could you be more specific?

Like assume you have some monitoring stack (kube-stack or grafana etc.) and either as a helm chart or as something else: What do you do when a CRD needs to be replaced? You write it in your deploy.sh script through some bash magic? Handling error cases in there?

What do you do when you want to see what you have deployed? Like checking if your test setup is rolled back? You always use kubectl to check some helm values? Or do you always annotate all pods with git commits which you then c&p?

A basic ArgoCD declarative setup takes perhaps a few hours if you do it for the first time, than you can give teams access to namespaces, they have log access, exec access if you provide a platform that is, in my opinion, quite a big advantage over just letting them use kubectl. Most developers i know do struggle with kubectl.

And for my own infra ( i also even setup argocd at home ) the dev cycle is much faster for me.

What "crappy cloud architecture" problem are you talking about specificly argocd apparently solves?

How do you test components? Like we have a test cluster and everyone uses argocd to just switch to a dev branch for a few days while testing. This works really well and you can switch back to main in argocd. And due to Argocd App Def you can even see all apps out of sync due to testing.

Re: We built the fastest CI and it failed

#286

I'm quite passioned about CI/CD and working in this field for the last 7 years. I'm keeping a close eye on jenkins, tekton, circleci, travis, github actions, gitlab etc. and never ever heard about earthly. Btw. one thing the person doesn't get: Everyone wants to have fast and reliable builds but no one cares. No one. CI/CD Systems in a developers eye are blackboxes. They don't care. And its already a solved problem s…

GitHub Actions is the same as everything else. It doesn't work any better or worse. But the UI is clunkier, you can do less with it than other systems, it lacked important features forever (still does). The only reason you think it's significantly better is psychological. The "market" of actions is better than competitors because the incumbent gets an advantage, and it trades on the idea of a generic executable as a…

I build enough pipelines in GitLab way to be able to tell you that GitHub Actions is much easier for me to implement.

Its much closer than if you would have a second system like jenkins, i already have access to git and the packages.

I do use much more container images than i did 6 or 10 years ago, that might have something to do with it but nonetheless, i do prefer GitHub Actions.

I also stoped liking GitLab though. The Autodevops was horrendes, the security issues is concerning and plenty of ideas or features are half done. this is also something which changed. I liked GitLab much more than i do now. But looking at the year, i haven't looked at GitLab for at least 2 years now.

Re: We built the fastest CI and it failed

#287
post #280

Earlier quoted context omitted.

> Being "fast" is not a selling point It could be, if that is a pain point for the customer. You could be 10x faster than any other option, but if "slow builds" are not on the top 10 list of problems they are dealing with, it's a meaningless statistic.

> It could be, if that is a pain point for the customer. But is it, though? I mean, are any of the competing CICD services noticeably slow, specially if we focus on the system's overhead? I don't think there are, at all. We're talking about at best millisecond differences in workflows that take minutes just to do the things we tell them to, such as setting up a build environment.

I'm only hearing about Earthly through this blog post, but is their optimization just shaving time off of overhead? From their landing page, it sounds like their system made caching and build targets a first class feature.

> Rebuild only what has changed, automatic parallel execution, automatic caching with no configuration required for the entire pipeline, no upload/download of cache – instantly available

You _could_ get these benefits on other providers like CircleCI, but it would require a lot more work/customization.

Re: We built the fastest CI and it failed

#288
post #147

Earlier quoted context omitted.

From my experience hosting our own instance of Gitlab + few CI/CD workers was few weeks of initial work (integrating with company's LDAP etc.) then around one ops-week per year worth of maintenance over last few years (50+ developers, dozens and dozens of projects). Before that we had just "a git server + jenkins + some slaves" that took even less maintenance (althought I'd imagine more fumbling on dev side with job…

You just described a great starting point for a small shop, I would recommend this route 100%. Initially your CI/CD needs to be close to the code. It needs to be close to developers. Not a DevOps department. It needs to be almost forgettable. Include a config in your repo, bam, welcome to the pipeline. Where this breaks down is when you don’t have LDAP, you have Azure IDP + Amazon’s Cognito + Keycloak 3rd Parties + S…

There are actually authentication providers that will pretend to be LDAP server for your apps, regardless of what auth backend you use.

But yeah, once your clients are big corporations you'd at least need to know how to implement some kind of cloud mess of authentication provider(s), still, doesn't need you need to infect your own stack with it.

Re: We built the fastest CI and it failed

#289

Earlier quoted context omitted.

Both GitHub and Slack have free tiers. Same principle, different software license.

It isn’t the same principle though. Slack and GitHub can sell data to make a profit etc but the problem with open source is people can just host it themselves meaning the creators don’t even have that option.

Plenty of organisations prefer to have managed options and thus pay for SaaS, even from open source vendors, rather than hosting their own.

Also you’re getting too caught up in specifics and missing the broader point. You want open source examples? How about Nexus, Redhat, nginx, prettier much all of Hashicorps offerings, basically half of why AWS resell.

The real issue with commercial open source isn’t restricting access to customers, it’s restricting access to resellers. As demonstrated with the examples above.

Re: We built the fastest CI and it failed

#290
post #51

Earlier quoted context omitted.

It's not about migrating the syntax. It's that people's CI over time become some kind of amalgamated model encapsulating how a firm makes every individual piece of software it makes and lands it places, think (ab)using a CI as Airflow (arbitrary automation job runner), and that the migration is first reverse engineering what people used to know, before even starting untangling all of that to express it some different…

I started work in December at an awesome company as a build/release engineer. Our current release workflow for our open-source components: 1. Run the CI build/test pipeline 2. If it passes, run the 'Publish' promotion (manually, on any pipeline you want) 3. The pipeline runs a make target 4. The make target starts up a VM (from the VM it's running on) 5. The make target SSH'es to the VM to run a make target 4. The ma…

Are you seriously considering moving from Makefile syntax to YAML?

Makefile syntax may have its problems, but nothing is as bad as YAML syntax.

YAML is worse than Makefile syntax on every criteria you may consider.

Post reply on HN