Building Out the SeatGeek Data Pipeline

#1

Post by dallasgutauckis » Fri, Jan 23, 2015, 5:44 PM UTC

Building Out the SeatGeek Data Pipeline
chairnerd.seatgeek.com

Re: Building Out the SeatGeek Data Pipeline

#2

Post by pythondan » Fri, Jan 23, 2015, 6:00 PM UTC

Do most people use Redshift for this sort of thing? Is that the best option out there?

Re: Building Out the SeatGeek Data Pipeline

#3

Post by dpods13 » Fri, Jan 23, 2015, 6:26 PM UTC

We've been using Redshift + Looker a lot lately at my company and it's been great. We've also added DynamoDb to our pipeline so that we're about to move data from DynamoDb -> Redshift -> Looker for running reports and analytics on NoSQL data

Re: Building Out the SeatGeek Data Pipeline

#4

Post by willcodeforfoo » Fri, Jan 23, 2015, 6:30 PM UTC

I've been interested in these database query frontend services/apps for ahwile (e.g., Looker, periscope, chartio) but every single one of them has only a Free Trial and no mention of pricing anywhere on their sites–-which makes me think its a multi-thousand dollar/month investment.

Is this true? If so, is there anything a little more cost-effective for running, visualizing, and saving database queries?

Re: Building Out the SeatGeek Data Pipeline

#5

Post by dpods13 » Fri, Jan 23, 2015, 6:48 PM UTC

willcodeforfoo wrote: post #4

I've been interested in these database query frontend services/apps for ahwile (e.g., Looker, periscope, chartio) but every single one of them has only a Free Trial and no mention of pricing anywhere on their sites–-which makes me think its a multi-thousand dollar/month investment. Is this true? If so, is there anything a little more cost-effective for running, visualizing, and saving database queries?

Yes, I believe Looker requires an annual contract at multi-thousand dollars a month. Not sure about the other products you listed and I don't know much about cheaper alternatives

Re: Building Out the SeatGeek Data Pipeline

#6

Post by [deleted] » Fri, Jan 23, 2015, 6:55 PM UTC

willcodeforfoo wrote: post #4

I've been interested in these database query frontend services/apps for ahwile (e.g., Looker, periscope, chartio) but every single one of them has only a Free Trial and no mention of pricing anywhere on their sites–-which makes me think its a multi-thousand dollar/month investment. Is this true? If so, is there anything a little more cost-effective for running, visualizing, and saving database queries?

[deleted]

Re: Building Out the SeatGeek Data Pipeline

#7

Post by maslam » Fri, Jan 23, 2015, 7:04 PM UTC

This is similar to our[1] pipeline. We love Luigi for what it brings to the table for building ETL pipelines.

[1] Appuri (www.appuri.com)

Re: Building Out the SeatGeek Data Pipeline

#8

Post by hglaser » Fri, Jan 23, 2015, 7:51 PM UTC

willcodeforfoo wrote: post #4

I've been interested in these database query frontend services/apps for ahwile (e.g., Looker, periscope, chartio) but every single one of them has only a Free Trial and no mention of pricing anywhere on their sites–-which makes me think its a multi-thousand dollar/month investment. Is this true? If so, is there anything a little more cost-effective for running, visualizing, and saving database queries?

Periscope co-founder here. We have plans as low as $350/mo. for startups with small datasets. I'm harry at periscope.io if that sounds appealing.

Re: Building Out the SeatGeek Data Pipeline

#9

Post by ajones » Fri, Jan 23, 2015, 9:17 PM UTC

pythondan wrote: post #2

Do most people use Redshift for this sort of thing? Is that the best option out there?

By this sort of thing do you mean using it as a data warehouse? That's what Redshift is branded as and the performance gains are definitely similar to what the blog outlines.

In my opinion, Redshift is the best data warehouse solution for a team building a small to medium-sized warehouse. This covers most use cases. For those building a data warehouse above a petabyte in size, you're going to have to look at different solutions. Redshift is powerful and relatively cheap compared to its competitors.