Slide 2&6, one query every 30 minutes. It turns out usage based billing can be cheaper if you don't use a resource.
I don't understand that part of the slides. Redshift isn't billed per-query, it's billed by instance-hour.
Amazon Redshift is 10x faster and cheaper than Hadoop and Hive
11–20 of 46 posts
Re: Amazon Redshift is 10x faster and cheaper than Hadoop and Hive
#12I am still new to large data, but isn't a solution like Redshift similar to Google's Big Query in that it only works with data that has a schema? How might one use Redshift with a db thats originally in Mongo?
Impose a schema.
Re: Amazon Redshift is 10x faster and cheaper than Hadoop and Hive
#13I wonder how it compares to BigQuery: https://developers.google.com/bigquery/docs/pricing
Re: Amazon Redshift is 10x faster and cheaper than Hadoop and Hive
#14Anyone interested in SQL queries on Hadoop should be checking out Cloudera Impala. It's open source.
Re: Amazon Redshift is 10x faster and cheaper than Hadoop and Hive
#151.2 TB really is not very much data in the context of "Big Data". The supposed advantage of Hadoop is that it can scale horizontally with linear performance.
Re: Amazon Redshift is 10x faster and cheaper than Hadoop and Hive
#16Are they benchmarking hash join on hadoop and redshift?
Re: Amazon Redshift is 10x faster and cheaper than Hadoop and Hive
#171.2 TB really is not very much data in the context of "Big Data". The supposed advantage of Hadoop is that it can scale horizontally with linear performance.
They note on slide 9 that this is only biggish data -- but sometimes, that's what you need to work with.
Re: Amazon Redshift is 10x faster and cheaper than Hadoop and Hive
#18They should compare redshift with hadoop + Imapala, OR hbase with Phoexnix from Salesforce. Comparing with hadoop + hive is not a correct comparison
Re: Amazon Redshift is 10x faster and cheaper than Hadoop and Hive
#19Disclaimer: I'm a committer on the Apache Hive project.
A couple points in no particular order:
* EMR Hive is a closed source fork of the upstream Apache Hive code base. The EMR docs imply that the latest version of EMR Hive is based on Apache Hive 0.8.1 (which was released more than a year ago), which means EMR users aren't benefitting from the performance improvements that appeared in the 0.9 and 0.10 releases.
* It is implied (though not explicitly stated) that the Hive queries were run against gzip compressed TSV files stored in S3, while Redshift was allowed to spend 17 hours converting the same data to its own optimized internal format. Hive supports an optimized columnar format too (RCFile). Why wasn't that used in this performance comparison?
Re: Amazon Redshift is 10x faster and cheaper than Hadoop and Hive
#20Disclaimer: I'm a committer on the Apache Hive project. A couple points in no particular order: * EMR Hive is a closed source fork of the upstream Apache Hive code base. The EMR docs imply that the latest version of EMR Hive is based on Apache Hive 0.8.1 (which was released more than a year ago), which means EMR users aren't benefitting from the performance improvements that appeared in the 0.9 and 0.10 releases. * I…
great feedback. I was also skeptical of using EMR hive due to the fact that it is so far behind in versions. Also RedShift can do the analytics part very well but I don't think it can do the exploration part that Hadoop/Hive are so good at (but maybe I am wrong)