Cloudera and Hortonworks merge

Post by Game_Ender » Thu, Oct 04, 2018, 11:57 AM UTC

jamesblonde wrote: post #46

HenryR wrote: post #26

Earlier quoted context omitted.

I am surprised that anyone is trying to differentiate on storage at this time, precisely when that's the part of the stack that's being cannibalized by the cloud vendors (look at the rate of innovation in HDFS over time; the effort is going elsewhere). Are you just targeting on-premise clusters, or is there some differentiation planned for the cloud as well?

We think that there is a niche for a higher performance dist FS than S3. We have integrated NVMe hardware with our HDFS implementation (HopsFS) and made its metadata layer distributed. NVMe means you can, for example, work with datasets with millions of files for deep learning - instead of having to munge them into parquet files because your FS is slowing down your machine learning pipeline. Reference: https://www.lo…

Post by alexnewman » Thu, Oct 04, 2018, 3:26 PM UTC

colin_mccabe wrote: post #45

alexnewman wrote: post #12

I was an early employee at Cloudera, am a Hadoop contributor and think this entire market is garbage. Basically the big data field went off the rails and this is Cloudera's way of trying to remain relevant. It's hard to point to a product that came after my tenure (I was on the team that original made the POS called Cloudera Manager) that's really used by anyone at scale. Cloud is displacing all of these tools and ne…

> It's hard to point to a product that came after my tenure (I was on the team that original made the POS called Cloudera Manager) that's really used by anyone at scale. There are tons of them. Spark, Hue, Sentry, Kafka, the list goes on.

Post by computronus » Thu, Oct 04, 2018, 6:30 PM UTC

coredog64 wrote: post #33

Tsarbomb wrote: post #27

Can someone explain to me what the big draw was for Hortonworks or Cloudera? Working as a lead in a small team that deals with a colossal amount of data (human genomics), it was always easier for us to hand roll deployments with terraform/ansible in either baremetal or OpenStack environments. In the public clouds like AWS we are using the managed services like EMR. The whole sales pitch I've always got from either ho…

You’re not wrong. It would be hard to sell CDH if you only engage with technical stakeholders. I can imagine the pitch now... “So what you get is really old versions of everything, plus some shaded jars that will break if your classloader ever changes load order. As far as cloud goes, we’ll give you this half-baked automation (Director) and we’ll also constrain you from using any features of your cloud provider (avai…

Post by purplezooey » Thu, Oct 04, 2018, 9:22 PM UTC

tumanian wrote: post #49

batoure wrote: post #42

Earlier quoted context omitted.

Don’t count MapR down, their commitment to developing ports and tools for the ecosystem outside the jvm was very forward thinking. Their internal engineering talent either makes them a prime aquaition target for CDH in future or a fierce competitor as systems like kudu become more common and production ready.

MapR is great while you are in their ecosystem - once you step outside and try to bring in technology thats not supported by them, you run into little showstopper bugs and complete lack of documentation and community support.

Post by alexnewman » Mon, Oct 08, 2018, 10:24 PM UTC

colin_mccabe wrote: post #57

dswalter wrote: post #51

Earlier quoted context omitted.

I think the parent comment was saying that none of these were primarily made by cloudera.

Hue is a Cloudera project, not even an Apache project. Sentry is an Apache project but primarily developed by Cloudera.