Viewing profile — epdlxjmonad

In this article, we report the results of evaluating the performance of the latest releases of Trino, Spark, Hive-MR3 using 10TB TPC-DS benchmark. Trino 476 (released in June 2025)…

Performance Evaluation of Trino 468, Spark 4.0.0-RC2, and Hive 4 on MR3 2.0 using the TPC-DS Benchmark In this article, we report the results of evaluating the performance of the f…

This article evaluates the performance of the following systems. Trino 418 (released on May 17, 2023) Spark 3.4.0 (released on Apr 13, 2023) Hive 3.1.3 on MR3 1.7 (released on May …

Diablo where you control a large party -- I tried this idea myself (using a bit of hack) and found it to be such great fun and extremely addictive. The only bad thing about it is t…

While I cannot give a definitive answer because I am not an expert on Spark internals, my opinion is that the discrepancy results mainly from query and runtime optimization. Apart …

Query plans are heavily optimized, and map-side joins are used extensively. The use of optimizations exploiting memory makes the so-called in-memory computing of Spark no longer re…

There is a common belief that SparkSQL is better than Hive because SparkSQL uses in-memory computing while Hive is disk-based. Another common belief is that Presto is better than H…

I agree that Spark on Kubernetes will have a hard time fixing the problem of shuffling. If they choose to use local disks for per-node shuffle service, a performance issue arises b…

Using a single thread to simulate everything is cool (as stated in my previous comment on FoundationDB at https://news.ycombinator.com/item?id=22382066 ). Especially if the overhea…

For debugging a distributed system, it may be just okay to use the traditional way consisting of testing, log analysis, and visualizing that everyone is familiar with. Yes, there a…