31–40 of 70 posts
Can Trino be used as a Snowflake replacement? How is the query speed compared to Snowflake?
Earlier quoted context omitted.
AWS Athena: selling a buggy, old, stale copy of someone else's work (Presto / Trino) for high prices and getting away with it because you control the platform. If that's not peak Amazon, I don't know what is.
Can’t you say the same thing for EC2 but with Linux instead of Presto? Personally I like Athena. The fact that it’s in the Amazon platform and managed is a plus for me.
I recently had to write SQL query generation for AWS Athena, which is based off Presto 0.217 It turns out that the dialect doesn't support LATERAL joins with a LIMIT in them. The below query only works if you remove the LIMIT clause. https://i.stack.imgur.com/rdB1s.png This makes saying things like "Fetch all artists where ..., for each artist fetch their first 3 albums where ..., and for each album fetch the top 10…
AWS Athena: selling a buggy, old, stale copy of someone else's work (Presto / Trino) for high prices and getting away with it because you control the platform. If that's not peak Amazon, I don't know what is.
It's one of the backends available in Splink, our FOSS record linkage software and it's revolutionary how it allows users to execute large scale probabilistic record linkage ridiculously cheaply. It wasn't long ago you needed very expensive proprietary software plus a big on prem cluster, costing in the hundreds of thousands, to achieve this.
A lot of the magic for me is on the infrastructure side: how they can read/write large datasets from s3 so quickly, so the value isn't just in the SQL engine.
Earlier quoted context omitted.
If you want to try an SaaS Athena alternative that's backed by Trino you can check out Starburst Galaxy: https://www.starburst.io/platform/starburst-galaxy/ Full disclosure I work at Starburst.
Oh nice, I have high opinions of you folks! Guy who goes by the name of "Randgalt" online builds some great Java libraries and works there too I believe.
Earlier quoted context omitted.
AWS Athena: selling a buggy, old, stale copy of someone else's work (Presto / Trino) for high prices and getting away with it because you control the platform. If that's not peak Amazon, I don't know what is.
I think many users just see they can execute a query on huge data cheaply and incredibly quickly and are delighted. That's certainly my experience. It's one of the backends available in Splink, our FOSS record linkage software and it's revolutionary how it allows users to execute large scale probabilistic record linkage ridiculously cheaply. It wasn't long ago you needed very expensive proprietary software plus a big…
One of the features I'm interested in (or would like to have) from Trino or Presto is the workload management which can better manage different types of queries and allocate resources accordingly. This becomes important when more applications adopt Trino or Presto as a distributed SQL database/platform, where the impact from different queries or workloads can be mitigated, besides the dedicated resources (CPU, MEM, e…
- https://engineering.salesforce.com/how-to-etl-at-petabyte-sc... - https://shopify.engineering/faster-trino-query-execution-inf... - https://trino.io/episodes/33.html - https://www.youtube.com/watch?v=-5mlZGjt6H4
All use the Lyft "Presto but really Trino"-Gateway project to run different clusters to handle various workloads. They go into various details for how this is achieved.
https://github.com/lyft/presto-gateway
Regarding the Trino/Presto split. I recommend looking at this blog to better understand why these two communities aren't mergeing. TL;DR Presto is a Facebook-driven project that mainly considers running on the Facebook infrastructure. Trino is community-driven that works on running well with all clouds and common infasturcture in the Trino community which is why you see a higher velocity there.
https://trino.io/blog/2022/08/02/leaving-facebook-meta-best-... https://trino.io/blog/2020/12/27/announcing-trino.html
Soon we anticipate that Trino will become the common name in the community space but we'll always love the origins of the Trino project being Presto.
Can Trino be used as a Snowflake replacement? How is the query speed compared to Snowflake?
Trino vs ClickHouse, can anyone tell from experience how those two compare?
I do the support for my department's trino cluster. We move ~1tb (and growing) in ETL jobs and support interactive queries for the data scientists/analysts. It would be super good if you guys added big query write support. Its really annoying to have to run a hive cluster in google to act as a proxy for this.
BigQuery very recently announced their Storage Write API which is one of the ways we were looking to implement this but there are some issues with the latency and consistency guarantees that it offers. But, yes, we do plan to add that eventually after ironing out all the kinks. See https://github.com/trinodb/trino/pull/13094