1.1B taxi rides benchmarked on distributed GPU-powered MapD
tech.marksblogg.com
1.1B taxi rides benchmarked on distributed GPU-powered MapD
1–10 of 23 posts
Re: 1.1B taxi rides benchmarked on distributed GPU-powered MapD
#2Amazing to see the improvements that MapD has made over the past few years. I have been following them for a long time, and was excited to catch wind of 3.0 this morning. Then I get on here to see someone already benchmarking and working with it.
Re: 1.1B taxi rides benchmarked on distributed GPU-powered MapD
#31.1B records = 500GB of raw CSV data.
This fits into RAM quite easily on a machine like the P2.8xlarge, especially when compression is used (like MapD uses).
I'd like to see how well this performs on a dataset that doesn't fit in the RAM.
Re: 1.1B taxi rides benchmarked on distributed GPU-powered MapD
#41.1B records = 500GB of raw CSV data. This fits into RAM quite easily on a machine like the P2.8xlarge, especially when compression is used (like MapD uses). I'd like to see how well this performs on a dataset that doesn't fit in the RAM.
Its honestly a pretty small dataset for us. MapD can easily do sub-100ms queries on 100B+ records with a single rack.
You can fit a lot of GPU + CPU memory on a small cluster. That RAM goes even further when you use compression like you mentioned.
We're fast at pulling data off of disk but we've never really aimed to be a traditional disk-based data warehouse. There are already great systems for that.
Re: 1.1B taxi rides benchmarked on distributed GPU-powered MapD
#5Is there a bridge for using MapD with Spark interface or somehow combining them? This can be interesting for the clusters with a lot of GPUs and a lot of data to do data manipulations.
Re: 1.1B taxi rides benchmarked on distributed GPU-powered MapD
#6Found elsewhere on the internet: 'On a system with eight Tesla K80s, which might cost somewhere between $60,000 to $70,000, the license for the MapD stack would be “a small multiple” of this hardware cost.'
I guess I'm not playing with this anytime soon.
Re: 1.1B taxi rides benchmarked on distributed GPU-powered MapD
#7Is there a bridge for using MapD with Spark interface or somehow combining them? This can be interesting for the clusters with a lot of GPUs and a lot of data to do data manipulations.
That actually sounds amazing, I would totally use that. I am actually hoping that they will create GDAL bindings for geospatial data.
Re: 1.1B taxi rides benchmarked on distributed GPU-powered MapD
#8Found elsewhere on the internet: 'On a system with eight Tesla K80s, which might cost somewhere between $60,000 to $70,000, the license for the MapD stack would be “a small multiple” of this hardware cost.' I guess I'm not playing with this anytime soon.
Still not cheap, but you could play with it on an p2.xlarge AWS instance for $4 an hour: https://aws.amazon.com/marketplace/pp/B01M0ZY2OV
Re: 1.1B taxi rides benchmarked on distributed GPU-powered MapD
#9Price comparison between Amazon Redshift, Google BigQuery, ElasticSearch and SlicingDice using the same dataset:
https://blog.slicingdice.com/slicingdice-pricing-model-and-c...
Re: 1.1B taxi rides benchmarked on distributed GPU-powered MapD
#101.1B records = 500GB of raw CSV data. This fits into RAM quite easily on a machine like the P2.8xlarge, especially when compression is used (like MapD uses). I'd like to see how well this performs on a dataset that doesn't fit in the RAM.
Its honestly a pretty small dataset for us. MapD can easily do sub-100ms queries on 100B+ records with a single rack. You can fit a lot of GPU + CPU memory on a small cluster. That RAM goes even further when you use compression like you mentioned. We're fast at pulling data off of disk but we've never really aimed to be a traditional disk-based data warehouse. There are already great systems for that.
Do you have any plans on offering MapD on Google Cloud Platform? (They have their own "Marketplace".)