Faster and more flexible pipelines with a Directed Acyclic Graph
about.gitlab.com
Faster and more flexible pipelines with a Directed Acyclic Graph
1–10 of 40 posts
Re: Faster and more flexible pipelines with a Directed Acyclic Graph
#2So, just like Concourse, Drone and any other modern CI system. About time.
Re: Faster and more flexible pipelines with a Directed Acyclic Graph
#3I didn't read too deeply, but I'm glad to see all these sorts of systems slowly converge on a sane model.
The part that saddens me is that they all end up looking like a glorified distributed make.
Re: Faster and more flexible pipelines with a Directed Acyclic Graph
#4I didn't read too deeply, but I'm glad to see all these sorts of systems slowly converge on a sane model. The part that saddens me is that they all end up looking like a glorified distributed make.
Why does that sadden you? Make doesn't do the job. Therefore a new tool is required. What's the problem?
Re: Faster and more flexible pipelines with a Directed Acyclic Graph
#5I didn't read too deeply, but I'm glad to see all these sorts of systems slowly converge on a sane model. The part that saddens me is that they all end up looking like a glorified distributed make.
The problem of DAG is solved by make very well. So well that it is why people insist on using it. In theory they would have made a good choice.
The issue is that "make" made sure to put enough traps so that people with less knowledge of the pitfalls tend to fall through. How many times I have seen recursive make calls without the $(MAKE) variable lead to funny races or lost compilation flags? How many paralelizable targets running in single thread because somebody did not understand some strange out of order execution(due to wrong Dependency mapping)? How many programs which are only partially built? How many projects were not actually built for huge periods of time because somebody forgot to put PHONY target? How many Frankenstein build systems exist which create Meta languages around make(kbuild) ? My latest adventure was a dormant fork bomb for 4 years in a daily updated project in the clean target! Whole development server farms being resource starved because the clean target started being executed as part of the normal build. How did the fork bomb happen? The usual recursive make.
Make is perfect actually,but we are not perfect in its usage.
Re: Faster and more flexible pipelines with a Directed Acyclic Graph
#6I'm confused, a DAG is a model for the relationships between data. What makes this a separate feature for the product? And it's pretty fundamental to the problem you try to solve, usually, so what were they using before, a list? Tree?
Re: Faster and more flexible pipelines with a Directed Acyclic Graph
#7I'm confused, a DAG is a model for the relationships between data. What makes this a separate feature for the product? And it's pretty fundamental to the problem you try to solve, usually, so what were they using before, a list? Tree?
Which part of the explanation of what they mean in the article is unclear?
Re: Faster and more flexible pipelines with a Directed Acyclic Graph
#8I'm confused, a DAG is a model for the relationships between data. What makes this a separate feature for the product? And it's pretty fundamental to the problem you try to solve, usually, so what were they using before, a list? Tree?
> I'm confused, a DAG is a model for the relationships between data.
Here they're talking about a DAG as a model for the relationships between computation rather than data.
Many systems that model computation use DAGs. For example it's how many compilers understand your code when you compile it.
Re: Faster and more flexible pipelines with a Directed Acyclic Graph
#9I'm confused, a DAG is a model for the relationships between data. What makes this a separate feature for the product? And it's pretty fundamental to the problem you try to solve, usually, so what were they using before, a list? Tree?
Which part of the explanation of what they mean in the article is unclear?
They're sort of coopting the name of a datastructure and using it as if it were a product name which is a bit weird
Re: Faster and more flexible pipelines with a Directed Acyclic Graph
#10I'm confused, a DAG is a model for the relationships between data. What makes this a separate feature for the product? And it's pretty fundamental to the problem you try to solve, usually, so what were they using before, a list? Tree?
Historically gitlab had a stages system: "Do all your builds, then all your tests, then all your deployments"