Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BlazingDB Origins, oh and we just raised $2.9M from Nvidia and Samsung

blog.blazingdb.com

Post reply on HN
Print view View original

31–38 of 38 posts

Re: BlazingDB Origins, oh and we just raised $2.9M from Nvidia and Samsung

#31

Post by felipe_aramburu »

arnon wrote: post #27
georgewfraser wrote: post #13

I would love to see performance numbers for a GPU data warehouse on a benchmark like TPCDS. My concern is that GPUs are very fast for a few operations, but that these operations aren't the bottleneck for realistic queries, so the overall performance will not be good. I did a TPCDS-based benchmark on Redshift-vs-Snowflake-vs-BigQuery a few weeks ago [1], it could probably be run by BlazingDB without too much trouble s…

TPC-DS has some incredibly complex queries that might be too complex for a database that doesn't have very extensive syntax coverage

I just glanced at about 15 of them and did not notice anything we did not support. There seems to be alot of nesting and whatnot but nothing here really jumps out at me. I have not looked at the full set yet.

Re: BlazingDB Origins, oh and we just raised $2.9M from Nvidia and Samsung

#32

Post by felipe_aramburu »

agibsonccc wrote: post #22
felipe_aramburu wrote: post #18

Earlier quoted context omitted.

People in the GPU space and slow storage systems like hadoop and S3) To caveat, I saw slow compared to something like storing your queryable data in system memory. The answer is that your clients are not normally GPU people. They are aware of GPUs and know they can be used for certain things but real clients normally have lots of data and they don't fire up a million instances to let it sit in a $10/GB storage soluti…

Great response thank you! I won't lie: We definitely have similar issues (I think any infra company runs in to this) Re: Kinetica. They are doing visualization as well as things like machine learning. I like your guys' approach a lot better. I see a clear bridge from the mass market commodity storage to something useful actually leveraging GPUs. That's why I commented. Best of luck to you!

Ok I see your point there. It does feel like that is quite a bit to manage. Our approach would be to perhaps try and show you how it could be used in these different kinds of workloads but would never try to undertake them ourselves. I'd probably sooner tear out my eyes than make data visualiation software.

Re: BlazingDB Origins, oh and we just raised $2.9M from Nvidia and Samsung

#33

Post by RHSman2 »

felipe_aramburu wrote: post #18
agibsonccc wrote: post #15

This actually looks pretty useful. I'm adjacent to you (DL on hadoop/spark competing more with the just launched amazon sagemaker among other things) and was curious about your go to market strategy. Like us you're kind of strattling the line between the hadoop ecosystem and GPUs. Mesos does this as well. It seems like most folks in the GPU space still don't get Hadoop or S3 as a data source yet though (despite it be…

People in the GPU space and slow storage systems like hadoop and S3) To caveat, I saw slow compared to something like storing your queryable data in system memory. The answer is that your clients are not normally GPU people. They are aware of GPUs and know they can be used for certain things but real clients normally have lots of data and they don't fire up a million instances to let it sit in a $10/GB storage soluti…

Well said

Re: BlazingDB Origins, oh and we just raised $2.9M from Nvidia and Samsung

#34

Post by felipe_aramburu »

ruw1090 wrote: post #26

What is the benefit of shipping data to the GPU for execution if the data is on S3 or HDFS? Won't most of the cost of the query be I/O?

Sure the very first time you run a query. But with multi tiered caching the data you frequently access sits closer and closer to the gpus so that alleviates that bottlekneck over time to an extent. Also what is a fantastic way of improving i/o? Compression and decompression. Our own file format compresses and decompresses using the GPU. We are working on doing the same for some of the Parquet decompression steps. I/O is almost always your main concern here, but you can improve upon it greatly by leverage processes that might not have been computationally feasible before.

Re: BlazingDB Origins, oh and we just raised $2.9M from Nvidia and Samsung

#35

Post by felipe_aramburu »

tmostak wrote: post #10

Congrats Rodrigo and team! We at MapD look forward to continued collaboration with you guys on the GOAI project and elsewhere!

Thanks bud. Looks like the GOAI people are making some moves! We are also happy to be sharing in that experience with you all.

Re: BlazingDB Origins, oh and we just raised $2.9M from Nvidia and Samsung

#37

Post by kwillets »

felipe_aramburu wrote: post #30
polskibus wrote: post #28

Earlier quoted context omitted.

Could you elaborate on the GPU + data lake part? Memory transfer lag to and from GPU is significant in comparison to GPU computing power. Data lake may mean mutliple heterogeneous data sources with or without schema. How is coping with it helped by a GPU?

So depending on your sources, e.g. if they are compressed or not, the GPU can greatly speed upi/o by compressing and decompressing directly on the GPU. This is particularly meaningful when you can store it in compressed state when you transfer to the GPU and decompress for use after its on. It doesn't solve most of the problems of working with hetergenous sources. That being said gpus definitely allowed us to speed u…

What types of compression do you use? I did some work on streamvbyte, and it seems relevant.

Re: BlazingDB Origins, oh and we just raised $2.9M from Nvidia and Samsung

#38

Post by felipe_aramburu »

kwillets wrote: post #37
felipe_aramburu wrote: post #30

Earlier quoted context omitted.

So depending on your sources, e.g. if they are compressed or not, the GPU can greatly speed upi/o by compressing and decompressing directly on the GPU. This is particularly meaningful when you can store it in compressed state when you transfer to the GPU and decompress for use after its on. It doesn't solve most of the problems of working with hetergenous sources. That being said gpus definitely allowed us to speed u…

What types of compression do you use? I did some work on streamvbyte, and it seems relevant.

Right now we support RLE, RLE Delta RLE, Delta RLE, Dictionary, Bitpacking. Many of these are combined together. It does look interesting I am checking it out.
Post reply on HN