NYT: Designing a Faster, Simpler Workflow to Build and Share Analytical Insights

#1

Post by thomson » Tue, May 23, 2017, 8:47 PM UTC

NYT: Designing a Faster, Simpler Workflow to Build and Share Analytical Insights
medium.com

Re: NYT: Designing a Faster, Simpler Workflow to Build and Share Analytical Insights

#2

Post by podopie » Tue, May 23, 2017, 8:55 PM UTC

Ah! This is me! Happy to take questions here or via email (edward.podojil@nytimes.com).

Re: NYT: Designing a Faster, Simpler Workflow to Build and Share Analytical Insights

#3

Post by thomson » Tue, May 23, 2017, 8:58 PM UTC

podopie wrote: post #2

Ah! This is me! Happy to take questions here or via email (edward.podojil@nytimes.com).

Any plans to open-source BQQL from the NYT? This looks like a really simple/lightweight DAG runner!

Re: NYT: Designing a Faster, Simpler Workflow to Build and Share Analytical Insights

#4

Post by podopie » Tue, May 23, 2017, 9:01 PM UTC

thomson wrote: post #3

podopie wrote: post #2

Ah! This is me! Happy to take questions here or via email (edward.podojil@nytimes.com).

Any plans to open-source BQQL from the NYT? This looks like a really simple/lightweight DAG runner!

I'd love to, but no plans currently. It was originally a hack week project that just turned large scale. We're looking into broadening the scope a bit first: adding parent/child relationships so there's better query dependency support, and long run looking to tie into AirFlow so analysts are using this UI but really creating AirFlow jobs in the background.