So much this. Another perfect example: https://databricks.com They just raised a $140m round of financing so apparently they have some good stuff going on. If you look at the website though: The Unified Analytics Platform. Accelerate innovation by unifying data science, engineering, and business. Sorry, what? Click on "learn more about the platform": DATABRICKS IS A TRULY UNIFIED APPROACH TO DATA ANALYTICS AT SCALE.…
(It's a Hadoop distribution).
Basically the product is Spark Notebooks (think Juypter) on AWS that allow you to quickly create clusters and even do fancy stuff with spark streaming.
A key thing that people miss also is the the creator of Spark is the CTO of Databricks and Databricks to some extent controls the direction of Spark. This probably impacts its valuation.