Live data from Hacker News

We built the fastest CI and it failed

earthly.dev

101–110 of 301 posts

Re: We built the fastest CI and it failed

#101
post #85
post #68

Earlier quoted context omitted.

You can self-host Gitlab CI runners and use them even in the free community version.

And honestly I can't even imagine ever using the public runners because they're permissionless. It's way easier to have them live in your own network and doling out permissions via tags that get picked up by different runners with different iam roles.

(mods: please forgive the off-topic personal question here)

Hi Spivak, You're not Lee Spivak (formerly of Upromise in the early 2000's), are you?

Re: We built the fastest CI and it failed

#102
post #87

Earlier quoted context omitted.

I know, this is what I’ve been doing for years. ( edit on GitHub, not GitLab, didn’t know GitLab allowed community CI now)

Then I guess I misunderstood "GitLab restricts CI/CD to paying customers"?

Sorry, I misread your comment as GitHub, not GitLab. I was unaware you can run CI runners on GitLab community version.

Re: We built the fastest CI and it failed

#103
You get customer when you solve a real problem for them. CI-Systems exists, slow or fast doesn't be a problem for most of the devs, cause you run normally everything local and only on publish in a remote CI-system. So CI-Systems exists and solve their problem.

I think only for a really small market the CI-performance is a real pain. Sure, everbody wants it fast, but that is not a reason to change a huge complex setup.

Re: We built the fastest CI and it failed

#104
post #85

Earlier quoted context omitted.

And honestly I can't even imagine ever using the public runners because they're permissionless. It's way easier to have them live in your own network and doling out permissions via tags that get picked up by different runners with different iam roles.

(mods: please forgive the off-topic personal question here) Hi Spivak, You're not Lee Spivak (formerly of Upromise in the early 2000's), are you?

This is why some people don’t leave the house.

Re: We built the fastest CI and it failed

#105
Not to rain on the parade here but this is literally a copy-pasta product.

Jenkins, Google Borg, Cloud Foundry, Concourse Pipelines...and surprise surprise when you look at where he came from...Ex-Google, Ex-VMW, RabbitMQ...

If OP wasn't in and around the source of all these tools above then they were at the very least first cousins to the story.

The sales cycle is long, integrations require multiple dimensions of executive buy-in, especially security and networking.

I think they made something nice but it felt like a nothing-burger story about something that is constantliy oscillating between bespoke and commoditized due to upstream problems that are such a mix of issues. Lack of oversight to micromanagement, inexperience to too much experience that they cannot let go of "the way it's always done".

It may be just my unpopular opinion but you are boiling an ocean of problems selling toolchains. Business and Tech are like water often finding the holes that lead to a path of least resistance, even when these erode the foundation of "core business". Toolchains that behave like guidelines and "parenting strategies" with removable guardrails have always offered the greatest rewards in my experience.

Re: We built the fastest CI and it failed

#106
post #62

This is a good write up of why you shouldn’t give away the house when you open source things. The issue was really this: Earthly being open-source, Earthy Satellite users were already seeing the benefit from 95% of Earthly CI . I’m a huge fan of open source, however, if your business model includes an open source model - you need a differentiator. Beyond blazingly fast(tm). You need a reason for people to offer up th…

I feel almost certain someone could wrap ArgoCD into a product if they handled the 'patternization' of applicationsets. D2iQ already do it with Flux, but we bailed on D2iQ before we even got to try it.

ArgoCD is almost worthless if your DevOps teams are IaC wielders. Honestly, just a bash script to push a container to the pods is all that’s required for DevOps deployment. A build pipeline should be get artifacts -> put artifacts in container -> push container to registry -> trigger a pull on your pods. If you didn’t have a standard cloud architecture, I can understand the frustration and reasoning behind looking at ArgoCD. If you have a standardized cloud practice, it’s moot.

Re: We built the fastest CI and it failed

#107

This is a good write up of why you shouldn’t give away the house when you open source things. The issue was really this: Earthly being open-source, Earthy Satellite users were already seeing the benefit from 95% of Earthly CI . I’m a huge fan of open source, however, if your business model includes an open source model - you need a differentiator. Beyond blazingly fast(tm). You need a reason for people to offer up th…

Did you mean argowf instead of argocd? Argocd is pretty simple and only syncs k8 resources.

Re: We built the fastest CI and it failed

#108
Sometimes the best technology fails and the space is overtaken by lesser technologies. It can be something as silly and out of your control as timing. Maybe the competition came out a bit after you and grew faster as interest in the space rose astronomically in parallel.

Re: We built the fastest CI and it failed

#109
post #41

Earlier quoted context omitted.

It's not about migrating the syntax. It's that people's CI over time become some kind of amalgamated model encapsulating how a firm makes every individual piece of software it makes and lands it places, think (ab)using a CI as Airflow (arbitrary automation job runner), and that the migration is first reverse engineering what people used to know, before even starting untangling all of that to express it some different…

I think the key is that nobody "wants" to spend any time at all on CI or their build system. It is one of those necessary evils that you have to do to have functioning software. Once you do it, even if it's a slow buggy pile of hacks, you will just ignore it until the pain gets very very bad. This was the same problem that caused our team to abandon CircleCI. They kept wanting us to rewrite our CI configs, touting va…

I had a talk with a circle CI rep at a conference and I tried to explain to him that yes I’m not happy with our custom Jenkins setup (agents, CasC, pipeline library), but that I also want to spend time and money on a migration project to end up at the status co again. If I would go to my boss and say we can either spend some time and fix our setup or spend time plus running costs for this external solution then the answer will be clear. I think convincing teams with huge stacks of CI setup to just switch because of marketing promises isn’t working. I would think of using a new setup when I start from scratch. At the moment at least.

Re: We built the fastest CI and it failed

#110

What does fast CI even mean? CI is an overgrown shell script running your build and telling you when it fails. In general build tooling has gotten so slow that the cost of whatever CI runner relative to it should be nil. If you want fast CI you need fast tsc, clang, rustc, etc... not a faster program that calls exec on them. A bit more on topic, if you're selling CI and your business fails it's because people you're…

> CI is an overgrown shell script running your build and telling you when it fails.

Oh, I wish it were that simple...

But, I also don't know what they measured. But, I'll give you just one example. Jenkins default landing page is a disaster in terms of speed. It tries to display data on recent builds everywhere across your cluster. For even not so big clusters this can be hundreds to thousands of items to pull from individual nodes running those builds. I've managed to crash Jenkins more times than I could count by simply loading the landing page w/o a particular configuration added that prevents it from its default behavior.

CI server will have its own database, typically, managing all sorts of CI entities: jobs, artifacts, users, secretes... It could be quite big and you'd do well taking care of proper indexing etc.

CI will have its assortment of runners, which are often provisioned dynamically (think VM or Docker images that need to be deployed to runner nodes), distributing them across the cluster is also not an easy task, if you want that to work fast. Of course you also want to be able to distribute the artifacts across the cluster -- this also takes time and resources.

CI will have its own bookkeeping necessary to, essentially, do garbage collection, reporting and self-diagnostic.

In big enough clusters this stuff can have a very high latency, if you don't specifically work towards reducing that latency.

> If you want fast CI you need fast tsc, clang, rustc, etc... not a faster program that calls exec on them.

Ever heard of ccache?

But, really, this is also super-naive. To make builds fast in a distributed system that CI is you would also need to figure out how to distribute this cache, how to modularize the builds... You might have heard about how one of the most difficult things in programming is cache invalidation? -- That's only partly a joke.

Post reply on HN