Photon powered Engine (enhance Apache Spark 3.0’s performance by up to 20x)
techcommunity.microsoft.com
Photon powered Engine (enhance Apache Spark 3.0’s performance by up to 20x)
1–3 of 3 posts
Re: Photon powered Engine (enhance Apache Spark 3.0’s performance by up to 20x)
#2For just a moment I was pleasantly surprised that I would be reading about an interesting propulsion technology. Seeing as that current propulsion approaches rely on ejecting Fermion composites at high relative velocities to the engine.
I am not pleasantly surprised anymore.
Re: Photon powered Engine (enhance Apache Spark 3.0’s performance by up to 20x)
#3DuxkDB's query engine is inspired by the same paper! You'll be surprised what you can process on a single node with DuckDB - it takes 33 Spark nodes to match performance! That still puts it ahead of Photon, and it is open source.