Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudera and Hortonworks merge

cnbc.com

Post reply on HN
Print view View original

11–20 of 60 posts

Re: Cloudera and Hortonworks merge

#11

Post by jamesblonde »

agibsonccc wrote: post #10
jamesblonde wrote: post #9

Earlier quoted context omitted.

We have a startup, Logical Clocks (www.logicalclocks.com), that just raised money to sell our data-science next-generation Hadoop stack. It has a new verison of HDFS called HopsFS (Nvme storage, distributed metadata) and support for GPUs in YARN. And distributed tensorflow, Spark, Airflow, Flink. So, what does that mean for us? I am shellshocked. I expect prices to increase (good for us). What else should we expect?

Both Horton and Cloudera require a huge number of partner resellers and consultants. What horton and cloudera have isn't just software, but brand, an install base, and the ability to back it up. They merged partially because they were both being cannibalized by services revenue they couldn't get rid of. Now they are struggling to move to the cloud (see Atlas now) I'm not sure yet another hadoop distro with a bunch of…

Adam, what about Java for deep learning? Is that needed?

Re: Cloudera and Hortonworks merge

#12

Post by alexnewman »

I was an early employee at Cloudera, am a Hadoop contributor and think this entire market is garbage. Basically the big data field went off the rails and this is Cloudera's way of trying to remain relevant. It's hard to point to a product that came after my tenure (I was on the team that original made the POS called Cloudera Manager) that's really used by anyone at scale. Cloud is displacing all of these tools and never got the clouds to play ball.

Re: Cloudera and Hortonworks merge

#13

Post by agibsonccc »

jamesblonde wrote: post #11
agibsonccc wrote: post #10

Earlier quoted context omitted.

Both Horton and Cloudera require a huge number of partner resellers and consultants. What horton and cloudera have isn't just software, but brand, an install base, and the ability to back it up. They merged partially because they were both being cannibalized by services revenue they couldn't get rid of. Now they are struggling to move to the cloud (see Atlas now) I'm not sure yet another hadoop distro with a bunch of…

Adam, what about Java for deep learning? Is that needed?

According to my customers and users yes? Granted, we do a lot more than just that though.

Dl4j itself has a decent sized user base. Ranging likely from your phone maker to your bank and retail store.

We have our own software distro too which is why I'm commenting on this. We don't try to boil the ocean with a bunch of tech though.

There's a whole new crop of companies focusing on solving bits of the ML problem well rather than trying to do storage and god knows what else.

My point here about you guys is you're trying to compete in what is largely a commodity market. People don't need all this stuff. Simplicity won here. It's not about better tech.

You guys have the same pitch MapR does and largely the same problem: Better tech is only part of the problem with adoption. You need customers, users, and a clear business model when going to market.

Cloudera and Horton ran one playbook that at least somewhat worked (it got them public) and now they can focus on competing with the cloud vendors, which made the right decision and just made commonly used software easy to use.

Re: Cloudera and Hortonworks merge

#14

Post by SpaceManNabs »

jamesblonde wrote: post #9
capkutay wrote: post #8

Good move for both companies. The surplus of 'enterprise hadoop' companies was created by a mixture of hype and a peak in VC investment in open source. The fact that Cloudera, Hortonworks, MapR were all founded and raised $100m+ around the same time was a bit superfluous for the whole market. Hard to say where this leaves MapR now. They seem to be the odd man out in terms of growth and adoption.

We have a startup, Logical Clocks (www.logicalclocks.com), that just raised money to sell our data-science next-generation Hadoop stack. It has a new verison of HDFS called HopsFS (Nvme storage, distributed metadata) and support for GPUs in YARN. And distributed tensorflow, Spark, Airflow, Flink. So, what does that mean for us? I am shellshocked. I expect prices to increase (good for us). What else should we expect?

You support Flink too? Nice! I am glad to see more and more people eyeing it as a viable framework for streaming.

Do you support Kubernetes too?

Re: Cloudera and Hortonworks merge

#15

Post by jamesblonde »

agibsonccc wrote: post #13
jamesblonde wrote: post #11

Earlier quoted context omitted.

Adam, what about Java for deep learning? Is that needed?

According to my customers and users yes? Granted, we do a lot more than just that though. Dl4j itself has a decent sized user base. Ranging likely from your phone maker to your bank and retail store. We have our own software distro too which is why I'm commenting on this. We don't try to boil the ocean with a bunch of tech though. There's a whole new crop of companies focusing on solving bits of the ML problem well r…

We don't. We are the only on-premise vendor with proper support for GPUs and python. We are a full data science stack, backed by Hadoop. We even have Kubernetes for model serving. And python in the cluster (with conda environments). And we have customers and funding. Nothing like MapR. And none of the legacy mapreduce crap.

Re: Cloudera and Hortonworks merge

#16

Post by jamesblonde »

SpaceManNabs wrote: post #14
jamesblonde wrote: post #9

Earlier quoted context omitted.

We have a startup, Logical Clocks (www.logicalclocks.com), that just raised money to sell our data-science next-generation Hadoop stack. It has a new verison of HDFS called HopsFS (Nvme storage, distributed metadata) and support for GPUs in YARN. And distributed tensorflow, Spark, Airflow, Flink. So, what does that mean for us? I am shellshocked. I expect prices to increase (good for us). What else should we expect?

You support Flink too? Nice! I am glad to see more and more people eyeing it as a viable framework for streaming. Do you support Kubernetes too?

Yes, but for model serving. Not for Flink or Spark. That's on YARN, for now.

Re: Cloudera and Hortonworks merge

#17

Post by agibsonccc »

jamesblonde wrote: post #15
agibsonccc wrote: post #13

Earlier quoted context omitted.

According to my customers and users yes? Granted, we do a lot more than just that though. Dl4j itself has a decent sized user base. Ranging likely from your phone maker to your bank and retail store. We have our own software distro too which is why I'm commenting on this. We don't try to boil the ocean with a bunch of tech though. There's a whole new crop of companies focusing on solving bits of the ML problem well r…

We don't. We are the only on-premise vendor with proper support for GPUs and python. We are a full data science stack, backed by Hadoop. We even have Kubernetes for model serving. And python in the cluster (with conda environments). And we have customers and funding. Nothing like MapR. And none of the legacy mapreduce crap.

You bundle way more than they do and on top of that have your own file system just like mapr does.

Your pitch is still about differentiated tech, not a large install base, a differentiated business model

and something related to people like a good partner ecosystem.

Your pitch here requires tons of services.

People don't know how to use all of this stuff especially on prem.

It takes more than just code to build a business.

I say this as someone who's been doing this since 2013. It's not easy.

Re: Cloudera and Hortonworks merge

#18

Post by jamesblonde »

agibsonccc wrote: post #17
jamesblonde wrote: post #15

Earlier quoted context omitted.

We don't. We are the only on-premise vendor with proper support for GPUs and python. We are a full data science stack, backed by Hadoop. We even have Kubernetes for model serving. And python in the cluster (with conda environments). And we have customers and funding. Nothing like MapR. And none of the legacy mapreduce crap.

You bundle way more than they do and on top of that have your own file system just like mapr does. Your pitch is still about differentiated tech, not a large install base, a differentiated business model and something related to people like a good partner ecosystem. Your pitch here requires tons of services. People don't know how to use all of this stuff especially on prem. It takes more than just code to build a bus…

Ok, now you're changing your angle. Differentiated tech is what we are all about - that is ok by me (for now).

If you want to train DNNs on a hundred GPUs today on-premise on TensorFlow, come to us, we can do it. They can't.

Re: Cloudera and Hortonworks merge

#20

Post by agibsonccc »

jamesblonde wrote: post #18
agibsonccc wrote: post #17

Earlier quoted context omitted.

You bundle way more than they do and on top of that have your own file system just like mapr does. Your pitch is still about differentiated tech, not a large install base, a differentiated business model and something related to people like a good partner ecosystem. Your pitch here requires tons of services. People don't know how to use all of this stuff especially on prem. It takes more than just code to build a bus…

Ok, now you're changing your angle. Differentiated tech is what we are all about - that is ok by me (for now). If you want to train DNNs on a hundred GPUs today on-premise on TensorFlow, come to us, we can do it. They can't.

I don't think I changed my angle here? I'm still addressing

the same point. Tech doesn't matter. Simplicity does.

Even in our own product line, we only do a small

subset of this. We don't even require a cluster

to run. We also work with tech that people use.

You are currently competing with horovod

and kubeflow. eg: "competing with free"

You need more than that to survive.

Generally, that comes down to services.

Post reply on HN