Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudera and Hortonworks merge

cnbc.com

Post reply on HN
Print view View original

31–40 of 60 posts

Re: Cloudera and Hortonworks merge

#31

Post by ewencp »

Tsarbomb wrote: post #27

Can someone explain to me what the big draw was for Hortonworks or Cloudera? Working as a lead in a small team that deals with a colossal amount of data (human genomics), it was always easier for us to hand roll deployments with terraform/ansible in either baremetal or OpenStack environments. In the public clouds like AWS we are using the managed services like EMR. The whole sales pitch I've always got from either ho…

terraform: 2014 ansible: 2012 openstack: 2010 cloudera: 2008 hortonworks: 2011 emr: 2011

(those were quickly googled, apologies if i got any dates wrong)

timing is important. cloudera had good, early timing and looked promising because of it. you are right that EMR definitely hurt all the other hadoop vendors, though i think people overestimate how comfortable big enterprises are with moving to public cloud. way more comfortable today; back then everything was a lot less certain. cloudera's name is unfortunate given they never got anything cloud-based successful, maybe they'd be in a better place now if they had.

but some of the key technologies you suggest as better options came 4-6 years later. that's 4-6 years of providing value, gaining traction, and building a committed customer base. 4-6 years is a long time, and even with how slow many enterprise projects run, more than enough to get entrenched, build tooling that makes a bunch of stuff easier, build mindshare, etc.

> Have I missed out on some cool stuff?

stuff that makes companies money doesn't always look cool.

Re: Cloudera and Hortonworks merge

#32

Post by PanosJee »

dfee wrote: post #25
dirkgently wrote: post #24

Earlier quoted context omitted.

> surplus Define surplus. Why having 3 or 4 companies in 1B plus market is surplus?

Because there are typically uneven returns due to the winner-takes-most style of internet businesses. Differentiation is of course key, then.

This is not an Internet business (no network effects), switching is not that hard on OSS stacks

Re: Cloudera and Hortonworks merge

#33

Post by coredog64 »

Tsarbomb wrote: post #27

Can someone explain to me what the big draw was for Hortonworks or Cloudera? Working as a lead in a small team that deals with a colossal amount of data (human genomics), it was always easier for us to hand roll deployments with terraform/ansible in either baremetal or OpenStack environments. In the public clouds like AWS we are using the managed services like EMR. The whole sales pitch I've always got from either ho…

You’re not wrong. It would be hard to sell CDH if you only engage with technical stakeholders. I can imagine the pitch now...

“So what you get is really old versions of everything, plus some shaded jars that will break if your classloader ever changes load order. As far as cloud goes, we’ll give you this half-baked automation (Director) and we’ll also constrain you from using any features of your cloud provider (availability zones, load balancers, Amazon Linux 2). Finally, we require you to use Oracle’s JDK but we won’t distribute it so that you’re indemnified.”

Re: Cloudera and Hortonworks merge

#34

Post by dikei »

Tsarbomb wrote: post #27

Can someone explain to me what the big draw was for Hortonworks or Cloudera? Working as a lead in a small team that deals with a colossal amount of data (human genomics), it was always easier for us to hand roll deployments with terraform/ansible in either baremetal or OpenStack environments. In the public clouds like AWS we are using the managed services like EMR. The whole sales pitch I've always got from either ho…

I have used both Ansible and and Ambari (from Hortonworks) to deploy and maintain our big data stack. They each have pros and cons

Ansible

Pros:

- More flexible

- Easier to update separate components

- Bring-your-own monitoring & alerting

Cons

- Higher initial time to setup playbooks

- Rolling update configuration is a pain

- Upgrading components require more compatibility testing

- Lack built-in monitoring & alerting

Ambari

Pros:

- Easier to setup from scratch (step-by-step wizard for everything)

- Changing configuration is easy, you change one components, and other components are update accordingly. It also notify you which software in which nodes need to be restarted and allow you to do rolling restart seamlessly.

- Components integrate well with each others, so less time is needed for compatibility testing.

- Back-ported of important patches .

- Built-in monitoring and alerting

Cons:

- Older software version, albeit with back-ported patches

- Cannot upgrade separated components

- Less easy to integrate with your existing monitoring infrastructure.

I also find it easier to train new team members to use an existing Ambari installation than to maintain Ansible playbooks. We are now using Ambari to maintain the more stable parts of our big data stacks (HDFS, YARN, etc), and use Ansible for the part which are still improving rapidly (Kafka, Flink, Presto, etc)

Re: Cloudera and Hortonworks merge

#36

Post by medh2000 »

nerpderp83 wrote: post #3

Yikes!

I have both used Cloudera stack and Hortonworks. I would say in my experience that installation and management is much straightforward with HW distribution. Both companies stacks have similar Apache projects and management software like Ambari and Cloudera Manager.

Re: Cloudera and Hortonworks merge

#37

Post by medh2000 »

thinkmassive wrote: post #28

> The two companies are committed to supporting existing offerings from the two companies for at least three years but will work on a "unity release" of software, drawing on technologies from both companies' portfolios, Reilly said. I’m really intrigued to see whether Ambari or Cloudera Manager wins out in the long run. This is unexpected but interesting. (I worked at Hortonworks 2014-2016, no current affiliation)

That's a good question. Both CM and Ambari are front end for managing all the services and more.

Re: Cloudera and Hortonworks merge

#39

Post by medh2000 »

threeseed wrote: post #35
Thaxll wrote: post #6

Hadoop very "fancy" 5-6 years ago, what's the trend now? I guess with managed services from AWS / Google it makes Hadoop less useful?

The managed data science services from AWS/Google is Hadoop/Spark.

True. Some one in the cloud is managing these Hadoop stacks for you. You just pay usage of these services.

Re: Cloudera and Hortonworks merge

#40

Post by batoure »

docker_up wrote: post #38

Lots of overlap between the two companies. I unfortunately expect a lot of layoffs on the Horton side of things, which is disappointing because I know quite a few people there.

I work for neither company but know people at both and think that this is highly inaccurate. Though there will be operational redundancy both companies make most of their revenue from support contracts and professional services since they provide a mostly FOSS product. Services and support are the two most heavily labor intensive parts of the tech industry every customer requires x amount of labor present. Additionally internal engineers paid to contribute to the FOSS products are still needed. Because both companies have been benifitting from the contributions of both teams of engineers for years. Since Horton has no closed source products it has very few redundant engineering resources.
Post reply on HN