Why hasn't Presto become industry standard?

if Presto SQL can query every database under the Sun (from RDBMS, Hadoop HDFS, Kafka and ElasticSearch) why it has not become industry standard yet? What's the catch? What are its limitation?

Lots of general reasons, inertia, etc. Often companies stick to 1-2 preferred technologies and adding Presto isn't seen as a gain (even though it helped Facebook quite a bit). I also suspect Amazon re-packing it as Athena reduced adoption to some extent.

If looking at Presto (now Trino), the main thing to keep in mind is that you inherit the limitations of the underlying data store.

Its best when the underlying store (+ the Db adaptor implementation) lets you parallelize work and keep each node busy, and avoid processing data unnecessarily. Hive/S3 columnar format data works great for this (IIRC this was a major early use case). Other sources like RDBMS will have natural limitations. Kafka has its own issues since each query generally means re-scanning a topic, etc.

I see the data bridges as most useful as a way to bring data into the native/optimal format. Then do the heavy lift work in Presto.

I've been using it for years with clients. It tends to sit in between source data and a final destination. Most data platforms are trying to take data from 10s if not 100s of sources and unify them. The variety of formats and sources is endless and I've often had to resort to using Python-based Airflow DAGs to collect data based on dates and store a cleaned up version of what was collected as a timestamped PQ file for Presto. Presto is then great for large scale transformations across the data and ad-hoc exploring.

Presto can be pointed at a lot of data stores but few external data providers offer ODBC-like interfaces. It seems to be either APIs or static file dumps for the most part. So Presto isn't going to be able to pull from these datasets alone.

In terms of security and maintenance, products like Redshift are easier to train traditional data warehouse people up on. The service is relatively cheap and has a nice UI for scaling.

The data world is extremely fragmented. Once firms have something in place changing it is going to be a struggle. Existing staff often gate keep and defend whatever technology they've staked their careers on. Once there are a lot of reports setup with any on data source migrating it could end up becoming a prolonged project which can be hard to sell.

It was quoted Snowflake had a $1M / day budget for sales and marketing. I'm not aware of any Presto consultancy spending that sort of money. Amazon does have Athena but they have countless other offerings which muddies the water.

@marklit,

Thanks for your insights. Great points on inertia and lack of big sales budgets. Agreed also on HDFC/data lake use cases with PQ files. However, regarding querying RDBMS, are you saying that Presto requires in ODBC/JDBC connectivity? Does Presto have an ability to connect with "native DB" drivers?

@gwittel, appreciate you sharing your insights. Will you be able to elaborate on "RDBMS will have natural limitations"? Can you provide a specific example?