Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

aws.amazon.com

Post reply on HN
Print view View original

81–90 of 111 posts

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#81

Post by unityByFreedom »

Abishek_Muthian wrote: post #80
unityByFreedom wrote: post #47

Earlier quoted context omitted.

They're literally keeping the source open. Call Amazon out for other stuff, not this. Enough with this good vs. evil attitude. People and companies are more complex than that and have varied interests. They do not fall neatly into one camp or the other.

When we say company we tend to subconsciously level them to any other business, But there's absolutely no way a company at the scale of Amazon could have got there without hiding some skeletons in the closet so it's good for the society in general to view ~~companies~~ Mega Corp. which has more money(read power) than several countries combined together always with suspicion.

Why not just call them out for the bad things and support the good things? Assuming everything will inevitably go bad at some point is (a) false and (b) misses the point: While Amazon carries the Open Search torch right now, at any point in the future everyone else can contribute.

They do deserve credit for stepping in to help even if it does support their own profit margins. Profit in itself isn't evil.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#82

Post by Abishek_Muthian »

unityByFreedom wrote: post #81
Abishek_Muthian wrote: post #80

Earlier quoted context omitted.

When we say company we tend to subconsciously level them to any other business, But there's absolutely no way a company at the scale of Amazon could have got there without hiding some skeletons in the closet so it's good for the society in general to view ~~companies~~ Mega Corp. which has more money(read power) than several countries combined together always with suspicion.

Why not just call them out for the bad things and support the good things? Assuming everything will inevitably go bad at some point is (a) false and (b) misses the point: While Amazon carries the Open Search torch right now, at any point in the future everyone else can contribute. They do deserve credit for stepping in to help even if it does support their own profit margins. Profit in itself isn't evil.

> Why not just call them out for the bad things and support the good things?

Not just calling them out, Even punishing them(fines) for their mistakes is not even a slap on their wrist at their scale we've seen this happening time and again with all Mega Corps.

There's just no incentive to not do something evil.

So extraordinary scrutiny for every single action seems to be the optimal approach until the governments decide to break Mega Corps. into smaller companies which can actually get hurt from punishments.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#83

Post by theamk »

binarymax wrote: post #44

So many people in this discussion take sides. The only thing I see, is divergence. APIs will now evolve separately, and compatibility will break soon. I think this competition and diversification will be good for innovation in search. But it’s also going to be confusing for a while. If you use Elasticsearch in Amazon, you better decide very quickly whether you want to hitch your wagon to the OpenSearch future. Migrat…

This didn't happen with MySQL/MariaDB split -- they are still mostly compatible.

Granted, it looks like Elastic is trying to break compatibility pretty hard (see the new version checks in clients for example) but hopefully Amazon will release universal connectors, it certainly in the Amazon's interests.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#84

Post by _msw_ »

nextaccountic wrote: post #79
_msw_ wrote: post #67

Earlier quoted context omitted.

Disclosure: I work for AWS where I build infrastructure services that sit low in the stack. I am a relative novice when it comes to Java-heavy codebases like Apache Lucene, Elasticsearch, or OpenSearch. But I know a thing or two about Open Source and Communities. I think it is hazardous to count the number of committers and PMC members that are part of a project that is built at the Apache Software Foundation. It is…

I think it's reasonable to expect that corporate citizens worth billions of dollars would do their fair share of open source contributions. I'm curious why you think this is bad for community health.

It is totally reasonable to expect for-profit corporations to be good stewards of shared resources, such as the public goods that make up the Free and Open Source Software (FOSS) commons. I think it is important to all users of FOSS that the software is well maintained, and is able to grow to meet the changing needs of users. We often call achieving a system where sufficient value created by FOSS can be captured in ways that funds its ongoing development and maintenance achieving a "self-sustaining" system. One view on sustainability in FOSS can be found in this Apache Software Foundation blog post [1].

Companies should also be respectful of the communities that produce and maintain software, including understanding the philosophy and governance that communities have adopted. For the Apache Software Foundation, this is being mindful to not unduly influence the project, such as to try to control it through employing a majority of the people who work on the project, or otherwise weaken the independence and autonomy for the community that builds and maintains the software. [2]

I've seen situations in the past where companies employ an open-source marketing strategy that ends up encouraging unhelpful activity in open source communities. This can also happen when companies measure the performance of a software developer based on metrics such as "number of patches accepted in an open source project." This kind of thing is generally unwelcome in open source software communities. See this thread on HN [3] on the Linux kernel development list, TL;DR - "Please don't waste maintainers' time on your KPI grabbing patches."

[1] https://blogs.apache.org/foundation/entry/the-apache-way-to-...

[2] http://community.apache.org/projectIndependence.html

[3] https://news.ycombinator.com/item?id=27629366

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#85

Post by deeblering4 »

unityByFreedom wrote: post #76
deeblering4 wrote: post #68

Earlier quoted context omitted.

I’m not so sure, they are also continuing to build a walled garden elk hosting alternative and already invested a lot of development into open distro for elasticsearch. People often run this software on their compute, so obviously eliminating proprietary license complications makes that an easier option for their customers to deploy, and redirects some of that licensing money into the customer’s aws spend. Don’t get…

You are not so sure that the world is not black and white? I assure you it's in color. Perhaps we do live in The Giver and only a few people realize this after all.

I agree, it's in color. In this case green.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#86

Post by DanielVZ »

EschatonCometh wrote: post #72
DanielVZ wrote: post #19

Earlier quoted context omitted.

We are doing the transition and everything is just so smooth. Adding new nodes is easy and fast. There's an actually good and consistent UI. Last time we tried to upgrade our cluster version on AWS we ended up waiting for weeks until it randomly updated mid day on a busy day and broke everything. It's a ton of small things that really make a difference.

You can upgrade instantly on AWS, it doesn't take weeks. A lot of problems with AWS Elasticsearch Service are caused by corporations trying to squeeze a dollar out of a dime. T2 and T3s with no Master Nodes, let alone three, leads to a lot of problems. When everything lives in MMAP'd memory and you are nickel and dime'n it with 4GB and one CPU. You're going to have a bad day.

Yeah that was for sure not the case for 3 m5.large master nodes and several r4.2xlarge nodes.

The update was instant, what wasn't instant was the wait for the update. IDK if it was the maintenance window (which was set up for a weekend) or some bug, but we DID end up waiting weeks until it randomly updated.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#87

Post by binarymax »

theamk wrote: post #83
binarymax wrote: post #44

So many people in this discussion take sides. The only thing I see, is divergence. APIs will now evolve separately, and compatibility will break soon. I think this competition and diversification will be good for innovation in search. But it’s also going to be confusing for a while. If you use Elasticsearch in Amazon, you better decide very quickly whether you want to hitch your wagon to the OpenSearch future. Migrat…

This didn't happen with MySQL/MariaDB split -- they are still mostly compatible. Granted, it looks like Elastic is trying to break compatibility pretty hard (see the new version checks in clients for example) but hopefully Amazon will release universal connectors, it certainly in the Amazon's interests.

Good point about MySQL/MariaDB. I think this is different though because search engines are at a big pivot point to include approximate-nearest-neighbor dense vector search (it has before been only sparse vector search for Lucene based platforms).

Specifically, this feature https://github.com/elastic/elasticsearch/issues/42326#issuec... will be a big change for Elasticsearch. OpenSearch might try to mimic the API, but implementation details here will matter a lot, since this type of search is really picky when it comes to performance/recall balance. OpenDistro has already been working on their own version: https://opendistro.github.io/for-elasticsearch/features/knn.... ...so will they switch their API? Perhaps - but the results are going to be very different.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#88

Post by gk1 »

binarymax wrote: post #87
theamk wrote: post #83

Earlier quoted context omitted.

This didn't happen with MySQL/MariaDB split -- they are still mostly compatible. Granted, it looks like Elastic is trying to break compatibility pretty hard (see the new version checks in clients for example) but hopefully Amazon will release universal connectors, it certainly in the Amazon's interests.

Good point about MySQL/MariaDB. I think this is different though because search engines are at a big pivot point to include approximate-nearest-neighbor dense vector search (it has before been only sparse vector search for Lucene based platforms). Specifically, this feature https://github.com/elastic/elasticsearch/issues/42326#issuec... will be a big change for Elasticsearch. OpenSearch might try to mimic the API, bu…

AFAIK when it comes to nearest-neighbor searches Elasticsearch is catching up with OpenDistro k-NN, and not the other way around.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#89

Post by binarymax »

gk1 wrote: post #88
binarymax wrote: post #87

Earlier quoted context omitted.

Good point about MySQL/MariaDB. I think this is different though because search engines are at a big pivot point to include approximate-nearest-neighbor dense vector search (it has before been only sparse vector search for Lucene based platforms). Specifically, this feature https://github.com/elastic/elasticsearch/issues/42326#issuec... will be a big change for Elasticsearch. OpenSearch might try to mimic the API, bu…

AFAIK when it comes to nearest-neighbor searches Elasticsearch is catching up with OpenDistro k-NN, and not the other way around.

Perhaps, but the point about divergence stands! Elastic won’t be copying the OpenDistro KNN API.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#90

Post by jka »

unityByFreedom wrote: post #47
atarian wrote: post #26

Earlier quoted context omitted.

>Amazon is the good guy This is exactly what Amazon wants you to think. That's why they're calling it "OpenSearch" and not "AmazonSearch".

They're literally keeping the source open. Call Amazon out for other stuff, not this. Enough with this good vs. evil attitude. People and companies are more complex than that and have varied interests. They do not fall neatly into one camp or the other.

Without a solid and forward-looking understanding of the internal cultures of both Elastic (the company) and also the specific team(s) at Amazon that will be managing the future of Elasticsearch and OpenSearch, respectively, I think it's too early to tell who will end up being the better steward of the search commons here.

Also worth noting that other open-source search engines exist out there; it's entirely possible that other alternative technologies will become more attractive, depending on how all this plays out.

Post reply on HN