I wish he would talk about how they protect one customer from running a query that brings down the full stack. When we permitted Tableau to start talking to Redshift, we frequently encountered "Oh crap, Peter is running that query and and that's why everything is at a stand-still..."
Why We Chose Redshift
11–20 of 45 posts
Re: Why We Chose Redshift
#12Redshift is like a prison, but with excellent accommodations. It's a great platform but it pretty much the perfect example of vendor lock-in.
Compared to the lock-in of the AWS ecosystem in general, Redshift honestly isn't that bad. You can unload all of your data into S3 and then do whatever you want with it. I'd be surprised if most data warehousing solutions had such an easy way of exporting the data.
In addition, if you store your data in S3 and have Redshift load it from there then you don't even need to do an export - just leave your source data in S3 after Redshift's loaded it, and you're all ready to switch to another platform.
Re: Why We Chose Redshift
#13I wish he would talk about how they protect one customer from running a query that brings down the full stack. When we permitted Tableau to start talking to Redshift, we frequently encountered "Oh crap, Peter is running that query and and that's why everything is at a stand-still..."
You can set up Workload Management[1] to restrict the amount of compute / query_slots each query/user can use.
It splits the memory/compute into slices, and queries can use multiple slices, so you can get some fine-grained control, but it takes a bunch of work.
[1] http://docs.aws.amazon.com/redshift/latest/dg/cm-c-modifying...
Re: Why We Chose Redshift
#14The title should say 'Amazon Redshift'. At first I thought its going to be about redshift vs f.lux: http://jonls.dk/redshift/ Edit: Why the downvote? redshift (and flux) exist since before 2010, whereas Amazon Redshift got introduced just in 2012. I think it is reasonable to assume that someone who has never heard of Amazon Redshift would think of the open source project first (that exists in various distributions as…
If we took a poll I suspect the majority would be thinking of the Amazon service - I know I was. The date the projects were introduced isn't necessarily relevant.
Re: Why We Chose Redshift
#15Redshift is like a prison, but with excellent accommodations. It's a great platform but it pretty much the perfect example of vendor lock-in.
Can you explain what you mean by that? I fail to see how a PostgreSQL query interface could possibly qualify as a perfect example of vendor lock-in.
If you want to move to another DW platform, it's probably not going to be Postgres-based. As every vendor has a slightly different flavor of SQL with different behaviors, this will require redesigning your queries, schemas, and most if not all of your stored procedures. Depending on the company and age of the platform, this could be many thousands of hours of work.
Really, vendor lock-in is pretty much a given with data warehousing platforms. Though these days, it's not uncommon for large companies to have multiple DW platforms all pulling data from each other. When one platform falls out of favor, the users just migrate themselves to another since most reporting systems not made by SAP or Oracle are compatible with pretty much everything.
Re: Why We Chose Redshift
#16The title should say 'Amazon Redshift'. At first I thought its going to be about redshift vs f.lux: http://jonls.dk/redshift/ Edit: Why the downvote? redshift (and flux) exist since before 2010, whereas Amazon Redshift got introduced just in 2012. I think it is reasonable to assume that someone who has never heard of Amazon Redshift would think of the open source project first (that exists in various distributions as…
If we took a poll I suspect the majority would be thinking of the Amazon service - I know I was. The date the projects were introduced isn't necessarily relevant.
The UI colorizer is what I thought of immediately, too.
If we took a poll I suspect the majority would be thinking of the Amazon service
That's just personal projection, and is as irrelevant as an argument beginning with, "I think most people would agree that..."
Personally, regardless of Amazon vs UI hack, I'm really tired of ambiguous naming in tech projects.
Re: Why We Chose Redshift
#17The title should say 'Amazon Redshift'. At first I thought its going to be about redshift vs f.lux: http://jonls.dk/redshift/ Edit: Why the downvote? redshift (and flux) exist since before 2010, whereas Amazon Redshift got introduced just in 2012. I think it is reasonable to assume that someone who has never heard of Amazon Redshift would think of the open source project first (that exists in various distributions as…
Same here, since I was recently using Redshift on an Ubuntu laptop after coming from F.lux on my Mac. Never heard of Amazon Redshift.
Re: Why We Chose Redshift
#18Earlier quoted context omitted.
Can you explain what you mean by that? I fail to see how a PostgreSQL query interface could possibly qualify as a perfect example of vendor lock-in.
If you want to move to another DW platform, it's probably not going to be Postgres-based. As every vendor has a slightly different flavor of SQL with different behaviors, this will require redesigning your queries, schemas, and most if not all of your stored procedures. Depending on the company and age of the platform, this could be many thousands of hours of work. Really, vendor lock-in is pretty much a given with d…
In contrast, Vertica, Greenplum, Netezza, Teradata Aster, and CitusDB are all based on PostgreSQL forks. In many cases, the client libraries behave like psql, and ease conversions at that level.
As to SQL language differences, no DW platform uses "standard SQL", just as no two RDBMS use the exact same SQL dialect.
I dare say database platform lock-in is a universal issue. Any migration will involve effort.
Re: Why We Chose Redshift
#19I wish he would talk about how they protect one customer from running a query that brings down the full stack. When we permitted Tableau to start talking to Redshift, we frequently encountered "Oh crap, Peter is running that query and and that's why everything is at a stand-still..."
*She :) I can understand why Ben Horowitz tries to default to using female pronouns.
Re: Why We Chose Redshift
#20That extra order of magnitude you pay in pricing you gain in response time.
Yeah, but a data warehouse isn't supposed to have great response times. Data warehouses are for large, low-value sets of historical data that you don't always know how you want to use. If you want to use data in real-time, you should be driving it from your transactional systems. Redshift and other data warehouse solutions are for doing reporting and dashboards, not triggering real-time reactions.
Well, used to be true, but now those systems are converging. -- Full disclosure, I work for a company working on exactly that problem called Treasure Data.