Dgsh – Directed graph shell

Post by wfunction » Mon, Jan 09, 2017, 9:21 PM UTC

cturner wrote: post #49

wfunction wrote: post #23

Earlier quoted context omitted.

Is it possible to show what XSLT is and why it's useful in like 5 minutes? I've always wanted a transformation language of some sort, but I've never managed to figure out XSLT (probably because I've never needed it) so I don't know what problems it solves or doesn't solve.

I've always wanted a transformation language of some sort, but I've never managed to figure out XSLT (probably because I've never needed it) so I don't know what problems it solves or doesn't solve. You'd be familiar with SQL. SQL is a declarative language for interacting with a relational structure. You say what you want and from where. It outputs to a table. Or, with significant effort, to more complex forms. XPath…

Post by samuell » Tue, Jan 10, 2017, 11:15 AM UTC

pptyp wrote: post #52

samuell wrote: post #28

Earlier quoted context omitted.

I have looked a bit at code examples of Airflow, but was worried that it seems to have a similar problem as a lot of other pipeline tools: That in the main workflow specification, dependencies are specified between tasks only, not between the individual inputs and outputs of each task (between tasks rather than data). This means that this info needs to be implemented "manually" in some less declarative manner somewhe…

We wrote a package to go with our Airflow installation to borrow some of the data flow (as opposed to Airflow's exclusive task deps flow you mention) concepts we liked from Make/Drake/Luigi. You may be interested: github.com/industrydive/fileflow