Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using Radicle CI

radicle.xyz

Post reply on HN
Print view View original

31–40 of 49 posts

Re: Using Radicle CI

#31

Post by everforward »

kuehle wrote: post #2

> I find the most frustrating part of using CI to be to wait for a CI run to finish on a server and then try to deduce from the run log what went wrong. I’ve alleviated this by writing an extension to rad to run CI locally: rad-ci. locally running CI should be more common

I've poked at this a few times, and I think it breaks down for CI that needs/wants to run integration tests against other services. Eg it wants to spin up a Postgres server to actually execute queries against.

Managing those lifetimes is annoying, especially when it needs to work on desktops too. On the server side, you can do things like spin up a VM that CI runs in, use Docker in the VM to make dependencies in containers, and then delete the whole VM.

That's a lot of tooling to do locally though, and even then it's local but has so many abstractions that it might as well be running in the cloud.

Re: Using Radicle CI

#32

Post by lbotos »

Op, Radicle had a very glitchy style home page before it went more 8-bit. Do you have an archive of that anywhere? I’d like to use it as reference for a style period in design!

Re: Using Radicle CI

#33

Post by RGBCube »

dan_manges wrote: post #28

There's a difference between small scale CI and large scale CI. Small scale: a project is almost small enough to run the build and tests locally, but you still want to have a consistent environment and avoid "works on my machine" problems. Large scale: a project is so large that you need to leverage remote, distributed computing to run everything with a reasonable feedback loop, ideally under 10 minutes. The opposite…

Sounds good, but it's still YAML and shell scripts. It's not even close to ideal.

A custom lazy, typed functional language that doesn't differentiate between expressions and "builds" would be much better. Even better if you add "contexts", aka implicit tags under values for automatic dependency inference. Also do "serializable bytecode", and closing over dependencies of thunks efficiently like Unison does for great distrubuted builds.

And it would be pretty easy to add a debugger to this system, same logic as doing "import"

Nix gets somewhat close, but it misses the mark by separating the eval and build phases. It having terrible documentation, 1332432 ways to do the same thing, not properly separating the nixpkgs/nix divide, and nixpkgs being horribly, but still insufficiently abstracted also doesn't help.

Also, I'm not sure why you posted this comment here, as there is nothing that prevents you from writing a Radicle CI adapter that can handle huge repositories. You can reference the bare git repo stored in the Radicle home, so you just need to be able to store the repo itself.

Re: Using Radicle CI

#34

Post by globular-toast »

kuehle wrote: post #2

> I find the most frustrating part of using CI to be to wait for a CI run to finish on a server and then try to deduce from the run log what went wrong. I’ve alleviated this by writing an extension to rad to run CI locally: rad-ci. locally running CI should be more common

Yeah, and I will never understand why developers accept anything less. GitHub CI is really bad for this. GitLab is a lot better as you can just run the exact thing through Docker locally. I like tools like tox too that automate your whole test matrix and do it locally just as well.

Re: Using Radicle CI

#35

Post by viraptor »

RGBCube wrote: post #33
dan_manges wrote: post #28

There's a difference between small scale CI and large scale CI. Small scale: a project is almost small enough to run the build and tests locally, but you still want to have a consistent environment and avoid "works on my machine" problems. Large scale: a project is so large that you need to leverage remote, distributed computing to run everything with a reasonable feedback loop, ideally under 10 minutes. The opposite…

Sounds good, but it's still YAML and shell scripts. It's not even close to ideal. A custom lazy, typed functional language that doesn't differentiate between expressions and "builds" would be much better. Even better if you add "contexts", aka implicit tags under values for automatic dependency inference. Also do "serializable bytecode", and closing over dependencies of thunks efficiently like Unison does for great d…

Every time there's yaml, you can use dhall and compile to json instead. You get typing and strictness that way - regardless of whether the service allows it internally.

Re: Using Radicle CI

#36

Post by 8n4vidtmkvmk »

girvo wrote: post #20
8n4vidtmkvmk wrote: post #10

Earlier quoted context omitted.

It does suck when those large scale integration tests fail but sometimes that's the only real way to test something. E.g. I have to call a service owned by another team. It has a schema and documentation so I can mock out what I think it will return, but how will I truly know the API is going to do what it says or what I think it says without actually calling the API?

Tbf that’s what post-deployment verification tests sre ideal for, instead of as integration/e2e tests blocking your merges/deployments

That's fine. If they're too slow, run them post deployment. But do run them at some point so you can at least catch it quickly without waiting for user complaints.

Re: Using Radicle CI

#37

Post by 8n4vidtmkvmk »

apwell23 wrote: post #12
8n4vidtmkvmk wrote: post #10

Earlier quoted context omitted.

It does suck when those large scale integration tests fail but sometimes that's the only real way to test something. E.g. I have to call a service owned by another team. It has a schema and documentation so I can mock out what I think it will return, but how will I truly know the API is going to do what it says or what I think it says without actually calling the API?

> I truly know the API is going to do what it says or what I think it says without actually calling the API? what if the API changes all of sudden in production? what about cases where api stays the same but content of response is all wrong? how do tests protect you from that? edit: they are not hypothetical scenarios. wrong responses are way more common than schema breaking. tooling upsteam is often pretty good at c…

That's exactly the point, isn't it? If the schema or response format changes, we want to catch that quickly. Canary, staging, or prod, take your pick but we need to call the API and run some assertions against that to make sure it's good.

Re: Using Radicle CI

#38

Post by cedws »

esafak wrote: post #18
kuehle wrote: post #2

> I find the most frustrating part of using CI to be to wait for a CI run to finish on a server and then try to deduce from the run log what went wrong. I’ve alleviated this by writing an extension to rad to run CI locally: rad-ci. locally running CI should be more common

dagger does this, at the expense of increased complexity.

I tried out Dagger hoping to tidy up some of our CI pipelines. I really wasn’t impressed to be honest. I initially just wanted to build a basic Docker image using BuildKit and even that was nightmare that the Dagger guys basically told me just isn’t supported.

Re: Using Radicle CI

#39

Post by cedws »

andrewaylett wrote: post #8
kuehle wrote: post #2

> I find the most frustrating part of using CI to be to wait for a CI run to finish on a server and then try to deduce from the run log what went wrong. I’ve alleviated this by writing an extension to rad to run CI locally: rad-ci. locally running CI should be more common

Hear, hear. Although I'd slightly rephrase that to "if you don't change anything, you should end up running pretty much the same code locally as in CI". GitHub Actions is really annoying for this, as it has no supported local mode. Act is amazing, but insufficient: the default runner images are huge , so you can't use the same environment, and it's not supported. Pre-commit on the other hand is fantastic for this kin…

I have to disagree about Act, my experience is that it only works for extremely simple workflows, and even then it’s easy to run into differences between Act and GitHub Actions. I’ve raised many bugs but AFAIK there’s like one guy working on it in his own time.

It’s terrible that the community has had to invent something like this when it should be provided by GitHub. I suspect the GitHub Actions team is a skeleton crew because nothing ever seems to get done over there.

Re: Using Radicle CI

#40

Post by esafak »

cedws wrote: post #38
esafak wrote: post #18

Earlier quoted context omitted.

dagger does this, at the expense of increased complexity.

I tried out Dagger hoping to tidy up some of our CI pipelines. I really wasn’t impressed to be honest. I initially just wanted to build a basic Docker image using BuildKit and even that was nightmare that the Dagger guys basically told me just isn’t supported.

When was this; dagger runs on top of buildkit?
Post reply on HN