Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a Data-Informed Culture

engineering.gusto.com

Post reply on HN
Print view View original

11–15 of 15 posts

Re: Building a Data-Informed Culture

#11

Post by dasternberg »

philipodonnell wrote: post #9
v64 wrote: post #8

Earlier quoted context omitted.

When I say raw data, I mean the data that's already been processed from original sources and stored in the data warehouse (that is, the warehouse data as it existed in our database and not how it was presented through third party tools like Tableau). I agree that for high level stuff like visits, etc, dashboards help out a lot. For some more specific examples where open access broke down: 1) We had a fact table with…

This is a great question and I hope there is more discussion around this. Our data group has similar issues with propagation of business rules to end users who want direct DW access but only know enough SQL to be dangerous.

Out of curiosity, are these end users building on top of each other's work, and if so what tools are they using to do this?

Re: Building a Data-Informed Culture

#12

Post by v64 »

dasternberg wrote: post #10
v64 wrote: post #8

Earlier quoted context omitted.

When I say raw data, I mean the data that's already been processed from original sources and stored in the data warehouse (that is, the warehouse data as it existed in our database and not how it was presented through third party tools like Tableau). I agree that for high level stuff like visits, etc, dashboards help out a lot. For some more specific examples where open access broke down: 1) We had a fact table with…

In my experience you have to strike a balance between allowing exploration that unblocks people outside the team to look into data (with a bit of a buyer-beware attitude) to help them build intuitions while making sure that core metrics that you use internally and/or share publicly are appropriately vetted. Using Aleph or other shared SQL tools like it allows us to quickly look at a query someone else on another team…

It sounds like we could've used something like Aleph to centralize the queries that were coming in, as opposed to the case where somebody copy and pastes a query to someone who adds a column and then emails it to someone else, etc etc. And you're right, there is no silver bullet, so it's always good to hear how other folks are making it work. Thanks for the response!

Re: Building a Data-Informed Culture

#13

Post by philipodonnell »

dasternberg wrote: post #11
philipodonnell wrote: post #9

Earlier quoted context omitted.

This is a great question and I hope there is more discussion around this. Our data group has similar issues with propagation of business rules to end users who want direct DW access but only know enough SQL to be dangerous.

Out of curiosity, are these end users building on top of each other's work, and if so what tools are they using to do this?

No, usually there is an enterprising analyst who knows a bit of SQL and wants to write a particular report, but doesn't want to wait for an IT cycle to free up. I'm the same way so I empathize. Most of the time it's Excel, because they all have it by default.

Re: Building a Data-Informed Culture

#14

Post by philipodonnell »

dasternberg wrote: post #10
v64 wrote: post #8

Earlier quoted context omitted.

When I say raw data, I mean the data that's already been processed from original sources and stored in the data warehouse (that is, the warehouse data as it existed in our database and not how it was presented through third party tools like Tableau). I agree that for high level stuff like visits, etc, dashboards help out a lot. For some more specific examples where open access broke down: 1) We had a fact table with…

In my experience you have to strike a balance between allowing exploration that unblocks people outside the team to look into data (with a bit of a buyer-beware attitude) to help them build intuitions while making sure that core metrics that you use internally and/or share publicly are appropriately vetted. Using Aleph or other shared SQL tools like it allows us to quickly look at a query someone else on another team…

Are you aware of any resources/perspectives that talk more about the balance between freedom to access data and the need to control the accuracy of what is taken out?

Re: Building a Data-Informed Culture

#15

Post by dasternberg »

v64 wrote: post #12
dasternberg wrote: post #10

Earlier quoted context omitted.

In my experience you have to strike a balance between allowing exploration that unblocks people outside the team to look into data (with a bit of a buyer-beware attitude) to help them build intuitions while making sure that core metrics that you use internally and/or share publicly are appropriately vetted. Using Aleph or other shared SQL tools like it allows us to quickly look at a query someone else on another team…

It sounds like we could've used something like Aleph to centralize the queries that were coming in, as opposed to the case where somebody copy and pastes a query to someone who adds a column and then emails it to someone else, etc etc. And you're right, there is no silver bullet, so it's always good to hear how other folks are making it work. Thanks for the response!

Aleph is an open source Rails app and some great folks where I used to work built it - worth a look: https://github.com/lumoslabs/aleph
Post reply on HN