OpenSearch: AWS fork of Elasticsearch and Kibana • Hacker News Board

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#61

Post by [deleted] » Mon, Apr 12, 2021, 5:50 PM UTC

z77dj3kl wrote: post #39

TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

My favorite "conspiracy theory" is that AWS intentionally creates stupidly verbose and numerous headers in all of their APIs just to up the bandwidth usage a few bytes per request at a time.

[deleted]

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#62

Post by bombcar » Mon, Apr 12, 2021, 5:50 PM UTC

andreygrehov wrote: post #17

Does that mean ElasticSearch documentation won't be relevant soon?

ElasticSearch will need to relatively quickly come out with a feature the OpenSearch doesn’t replicate or people will just use the minimum that both support (see MySQL vs MariaDB).

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#63

Post by Dylan16807 » Mon, Apr 12, 2021, 5:55 PM UTC

busterarm wrote: post #36

real_joschi wrote: post #22

Earlier quoted context omitted.

If you're not using any of the AWS services, that might be true but then you're also leaving a lot of potential on the table. If you're "cloud-agnostic" and could migrate away from AWS in the blink of an eye then you're paying for an overpriced VM offering and should probably migrate to a cheaper hosting provider immediately.

> If you're "cloud-agnostic" and could migrate away from AWS in the blink of an eye then you're paying for an overpriced VM offering and should probably migrate to a cheaper hosting provider immediately. No this is what everyone suggesting this does not get. The offerings are not equivalent. There are a baseline of services that the cloud providers offer that can be made functionally-equivalent. It's not just EC2 but…

> all of these available within the same VPC (talking to each other without paying bandwidth costs).

That's one of the real key issues, I think.

If bandwidth is 10x cheaper, then maybe you decide it's fine if only half your inter-server bandwidth is free.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#64

Post by retzkek » Mon, Apr 12, 2021, 5:56 PM UTC

phd514 wrote: post #4

From the announcement: "You should consider the initial code to be at an alpha stage — it is not complete, not thoroughly tested, and not suitable for production use. We are planning to release a beta in the next few weeks, and expect it to stabilize and be ready for production by early summer (mid-2021)." Given that Amazon announced the fork in January and they don't expect it to be production-ready until summer, I'…

> they've underestimated the amount of work required to package and distribute a product as complex as Elasticsearch

The bulk of the work thus far has been to strip out the non-OSS components ("X-pack") and the many references to it, nothing to do with packaging, distributing, or even maintaining and developing features.

https://discuss.opendistrocommunity.dev/t/preparing-opensear...

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#65

Post by GordonS » Mon, Apr 12, 2021, 5:56 PM UTC

martinald wrote: post #54

GordonS wrote: post #49

Earlier quoted context omitted.

It really is price gouging - bandwidth is actually cheap. A couple of comparisons: Oracle Cloud give you 10TB of bandwidth for free, with overage charged at around €7.5/TB. You can rent a VPS from the likes of Hetzner, and they will throw in 2-20TB of bandwidth for free, with overage charged at something like €1/TB - AWS charge an eye-watering €125 for each TB! I think the reason the big 3 (AWS, Azure, GCP) still cha…

I totally agree, but was stunned to see DigitalOcean charging $0.10/GB for outbound transfer for their new (quite cool imo) apps service. You do get 40GB-100GB included, but it means it's unusable for bandwidth heavy apps. They include 1TB (which is pooled across all VMs) and 0.01/GB on their standard platform.

Yeah, I think Digital Ocean are really taking the piss there - proven by their existing $0.01/GB pricing, as you mentioned.

I reckon DO have seen the big providers getting away with it, and they want a piece. Why wouldn't they, when there is apparently so little pushback?

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#66

Post by randoramax » Mon, Apr 12, 2021, 5:56 PM UTC

ampdepolymerase wrote: post #56

TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

What's worse is the second order repercussions. Future open core/open source SaaSes will go straight to something like the Business Source License or the MongoDB license instead of traditional libre licenses. Amazon has done an incredible amount of damage to the open ecosystem.

Good. So the VCs will go back to playing like in the old days of shareware software licenses. It'll be good.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#67

Post by oever » Mon, Apr 12, 2021, 5:56 PM UTC

The new name clashes with the Open Search Foundation.

https://opensearchfoundation.org/

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#68

Post by ignoramous » Mon, Apr 12, 2021, 5:57 PM UTC

OpenSearch was once was an initiative founded at A9, Amazon subsidiary, to create a personalized, cross-service, search engine: https://archive.is/PCKWq

OpenSearch is from an era when Amazon and Google were covertly competitive. Google didn't get anywhere with Froogle and AppEngine; whilst Alexa and A9 didn't move any mountains.

Code: https://github.com/dewitt/opensearch

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#69

Post by williesleg » Mon, Apr 12, 2021, 5:57 PM UTC

Oh this will not end well.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#70

Post by TechBro8615 » Mon, Apr 12, 2021, 5:59 PM UTC

prepend wrote: post #29

TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

> In fact, Amazon contributes very little to open source in general, considering how much they take from it I don’t think this is a fact. Amazon seems to contribute pretty significantly according to the pages [0,1] they put out that describes their contributions. Not to mention their membership in OSS foundations like Linux Foundation. [2] You have the caveat about in relation to benefit they gain, but that’s pretty…

The absolute claim, aside from being at home in a rant on HN, comes from a cursory glance at https://github.com/amzn, weighted by contributors and popularity, and compared to companies of similar size. Google, Microsoft and Facebook all build and maintain multiple open source projects that are hugely popular with people who use them outside of the company sandboxes. For example, people benefit from React without Facebook gaining much directly. (Facebook! If Facebook has any redeeming qualities, it's their open source contributions to the frontend ecosystem, although I promise you I could ascribe malicious intent to those as well...) Contrast that with Amazon. On their GitHub page, I see a few obscure projects amongst a bulbous array of AWS SDKs.

To the sibling comment that asked about Firecracker -- I think Firecracker is awesome, and I did mention that in my original complaint. They even created it themselves! Well, a team of amazing engineers in Romania did. I have no personal insight into the matter, but it seems like they operate relatively independently from the AWS profit machine. Good for them too, it's incredible software. But I'm sure if they were to tell the story of how they got buy-in at Amazon to open source it, the same themes would come up -- how does Amazon benefit from this? In the case of Firecracker, the more people test it / harden it / run Doom on it, the more value Amazon can provide on its serverless platform. So again, unlikely to be purely altruistic intentions... but that's not to say there's anything wrong with that. I just find it all a bit distasteful in aggregate.