OpenSearch: AWS fork of Elasticsearch and Kibana

From the announcement: "You should consider the initial code to be at an alpha stage — it is not complete, not thoroughly tested, and not suitable for production use. We are planning to release a beta in the next few weeks, and expect it to stabilize and be ready for production by early summer (mid-2021)."

Given that Amazon announced the fork in January and they don't expect it to be production-ready until summer, I'm guessing they've underestimated the amount of work required to package and distribute a product as complex as Elasticsearch. Given that, I doubt they will be well-equipped to keep pace with new feature development.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#5

Post by lornajane » Mon, Apr 12, 2021, 4:36 PM UTC

phd514 wrote: post #4

From the announcement: "You should consider the initial code to be at an alpha stage — it is not complete, not thoroughly tested, and not suitable for production use. We are planning to release a beta in the next few weeks, and expect it to stabilize and be ready for production by early summer (mid-2021)." Given that Amazon announced the fork in January and they don't expect it to be production-ready until summer, I'…

I'm not sure I agree with that assessment. Now that the fork is publicly available, others can contribute to get it ready, which wasn't possible until now.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#6

Post by busterarm » Mon, Apr 12, 2021, 4:36 PM UTC

phd514 wrote: post #4

From the announcement: "You should consider the initial code to be at an alpha stage — it is not complete, not thoroughly tested, and not suitable for production use. We are planning to release a beta in the next few weeks, and expect it to stabilize and be ready for production by early summer (mid-2021)." Given that Amazon announced the fork in January and they don't expect it to be production-ready until summer, I'…

Given how poorly of a job Elastic themselves did with keeping the full ecosystem of tools working in lockstep for YEARS, I'm sure Amazon will do fine.

I remember all through Elasticsearch 5 where none of their packaged Kibana dashboards flippin' worked.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#7

Post by colechristensen » Mon, Apr 12, 2021, 4:36 PM UTC

phd514 wrote: post #4

From the announcement: "You should consider the initial code to be at an alpha stage — it is not complete, not thoroughly tested, and not suitable for production use. We are planning to release a beta in the next few weeks, and expect it to stabilize and be ready for production by early summer (mid-2021)." Given that Amazon announced the fork in January and they don't expect it to be production-ready until summer, I'…

I would assume they’re doing more than just packaging.

A beginning is the time for taking the most delicate care that the balances are correct

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#8

Post by phd514 » Mon, Apr 12, 2021, 4:39 PM UTC

lornajane wrote: post #5

phd514 wrote: post #4

From the announcement: "You should consider the initial code to be at an alpha stage — it is not complete, not thoroughly tested, and not suitable for production use. We are planning to release a beta in the next few weeks, and expect it to stabilize and be ready for production by early summer (mid-2021)." Given that Amazon announced the fork in January and they don't expect it to be production-ready until summer, I'…

I'm not sure I agree with that assessment. Now that the fork is publicly available, others can contribute to get it ready, which wasn't possible until now.

Yes, others can contribute, but significant feature development on large-scale OSS projects tends to be driven by developers paid to work on the project full-time and coordinated by an organized steering committee with clear governance (or company if the product is owned by a single company). I don't see any of that in place for OpenSearch and getting that all started up is not at all a trivial endeavor.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#9

Post by TechBro8615 » Mon, Apr 12, 2021, 4:41 PM UTC

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully?

From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / console templates. Indeed, most of this is templated -- AFAIU, for example, each AWS service autogenerates its Boto bindings and parts of its console frontend via code generators. Amazon has really mastered the factory process of churning out new services, and when they find a popular one, they can invest more resources into developing it than the original team ever could.

And therein lies the rub. If Amazon is improving the software in a way that the original team couldn't, it's hard to say that the community isn't benefiting. I think what strikes me the wrong way is that Amazon is not doing it for any altruistic reason. In fact, Amazon contributes very little to open source in general, considering how much they take from it. Compare them to Facebook (React, etc) or Google (tons of dev tools) or Microsoft (VSC, TypeScript). What does Amazon have? Firecracker, kind of? And now a fork of ES because that's the only way they could continue making money off it without violating the license a small startup put in place to stop them?

Well, good for Amazon, I suppose, but I find myself instinctively disliking them for this. I'm not sure what the solution is. Hopefully technologies like Kubernetes and Terraform will encourage big customers to become at least cloud-agnostic, if not cloud-independent. At the very least it would be great if Amazon / Google / Microsoft stopped gouging bandwidth at such absurd margins. Or not. Maybe it will be their downfall as startups differentiate along those lines. That would be ironic, coming from the originators of "your margin is my opportunity."

Personally I'm doing my part by not building anything with vendor lock-in. It's great to be able to deploy to any cloud, if you value either robustness or flexibility.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#10

Post by busterarm » Mon, Apr 12, 2021, 4:42 PM UTC

TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

As a longtime ElasticSearch cluster admin/developer and Elastic Cloud customer, I don't feel bad for Elastic in the slightest and I'm psyched about this fork.

The way they operate their cloud service leaves a lot to be desired and encourages maximum spend if you end up wanting to use it for anything demanding in production.