Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

aws.amazon.com

Post reply on HN
Print view View original

91–100 of 111 posts

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#91

Post by _msw_ »

binarymax wrote: post #89
gk1 wrote: post #88

Earlier quoted context omitted.

AFAIK when it comes to nearest-neighbor searches Elasticsearch is catching up with OpenDistro k-NN, and not the other way around.

Perhaps, but the point about divergence stands! Elastic won’t be copying the OpenDistro KNN API.

It is pretty clear to me that Elastic is planning to build their ANN features differently than OpenDistro's k-NN implementation, or other plugins modules that extend Easticsearch in similar ways. They now will build on the Apache Lucene capabilities that were collaboratively built "upstream" by a number of individuals, some that work for Amazon and some that work for Elastic.

From the linked issue, it seemed that they were originally planning to develop this as a proprietary feature of Elasticsearch, without contributing the functionality to Apache Lucene, but then changed direction when the Apache Lucene developers (some of which are currently employed to do such work by Amazon) started to build its approximate nearest neighbor (ANN) vector search capabilities. [1]

It's great to see folks that work for Elastic collaborating and building on what is in Apache Lucene to extend the utility of ANN with Hierarchical Navigable Small World Graphs (HNSW) [2]! From this, I think it should be possible to implement an Open Source version of the functionality with a compatible API, if that is something that OpenSearch users seek.

[1] https://issues.apache.org/jira/browse/LUCENE-9004

[2] https://github.com/apache/lucene/pull/250

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#92

Post by nextaccountic »

_msw_ wrote: post #84
nextaccountic wrote: post #79

Earlier quoted context omitted.

I think it's reasonable to expect that corporate citizens worth billions of dollars would do their fair share of open source contributions. I'm curious why you think this is bad for community health.

It is totally reasonable to expect for-profit corporations to be good stewards of shared resources, such as the public goods that make up the Free and Open Source Software (FOSS) commons. I think it is important to all users of FOSS that the software is well maintained, and is able to grow to meet the changing needs of users. We often call achieving a system where sufficient value created by FOSS can be captured in w…

Indeed commit count is a silly metric. I reminds me of the bad incentives of the hacktoberfest

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#93

Post by foota »

tedivm wrote: post #62
true_religion wrote: post #42

Earlier quoted context omitted.

The fact that they grew despite Amazon doesn't mean that AWS Elasticsearch wasn't a viable competitor. It just means that at the moment it was not an existential threat. However, when competing with a literally A in FAANG, one must be very careful about writing them off. To compare, I knew a company that 10 years ago made really great business intelligence integrations into Excel. They'd operated for 15+ years and we…

I'd have more sympathy for that viewpoint if Elastic didn't start off by using the open source community before switching to a proprietary license. The never would have gotten to the level where they mattered if it wasn't for being open source, and turning their back on that community makes them the bad guys.

Isn't elastic still open source, just not in the way the open source diehards like?

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#94

Post by unityByFreedom »

deeblering4 wrote: post #85
unityByFreedom wrote: post #76

Earlier quoted context omitted.

You are not so sure that the world is not black and white? I assure you it's in color. Perhaps we do live in The Giver and only a few people realize this after all.

I agree, it's in color. In this case green.

That's just as myopic. We all do many things daily with no expectation of monetary reward.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#95

Post by unityByFreedom »

jka wrote: post #90
unityByFreedom wrote: post #47

Earlier quoted context omitted.

They're literally keeping the source open. Call Amazon out for other stuff, not this. Enough with this good vs. evil attitude. People and companies are more complex than that and have varied interests. They do not fall neatly into one camp or the other.

Without a solid and forward-looking understanding of the internal cultures of both Elastic (the company) and also the specific team(s) at Amazon that will be managing the future of Elasticsearch and OpenSearch, respectively, I think it's too early to tell who will end up being the better steward of the search commons here. Also worth noting that other open-source search engines exist out there; it's entirely possible…

> Without a solid and forward-looking understanding of the internal cultures of both Elastic (the company) and also the specific team(s) at Amazon that will be managing the future of Elasticsearch and OpenSearch, respectively, I think it's too early to tell who will end up being the better steward of the search commons here.

Amazon is the better steward of open sourced Elastic Search, hands down, because nobody else is doing it. I'm not even sure that stewardship of the technology's proliferation is TBD. Didn't Elastic close the source because they feared Amazon was taking too much of their potential revenue? Elastic gambled that only they could carry the technology forward, and they lost that bet. I'll grant it's possible Elastic could become more competitive with Amazon+OpenSearch somehow down the road, however it appears that right now Elastic is on its heals while Amazon is driving forward. It's also possible that another closed-source search engine becomes popular. That said, Elastic is built atop Lucene, and as long as an open sourced version of Elastic exists I'm not sure why you would start from scratch or with something else.

> Also worth noting that other open-source search engines exist out there; it's entirely possible that other alternative technologies will become more attractive, depending on how all this plays out.

Which open-source search engines do you think are competitive?

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#96

Post by unityByFreedom »

Abishek_Muthian wrote: post #82
unityByFreedom wrote: post #81

Earlier quoted context omitted.

Why not just call them out for the bad things and support the good things? Assuming everything will inevitably go bad at some point is (a) false and (b) misses the point: While Amazon carries the Open Search torch right now, at any point in the future everyone else can contribute. They do deserve credit for stepping in to help even if it does support their own profit margins. Profit in itself isn't evil.

> Why not just call them out for the bad things and support the good things? Not just calling them out, Even punishing them(fines) for their mistakes is not even a slap on their wrist at their scale we've seen this happening time and again with all Mega Corps. There's just no incentive to not do something evil. So extraordinary scrutiny for every single action seems to be the optimal approach until the governments de…

They're a huge company. If you want to attack you need to be able to identify where they are acting badly and strike at that weakness. Calling the whole thing an evil empire isn't going to make any progress: your attacks are easily fended off.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#97

Post by Abishek_Muthian »

unityByFreedom wrote: post #96
Abishek_Muthian wrote: post #82

Earlier quoted context omitted.

> Why not just call them out for the bad things and support the good things? Not just calling them out, Even punishing them(fines) for their mistakes is not even a slap on their wrist at their scale we've seen this happening time and again with all Mega Corps. There's just no incentive to not do something evil. So extraordinary scrutiny for every single action seems to be the optimal approach until the governments de…

They're a huge company. If you want to attack you need to be able to identify where they are acting badly and strike at that weakness. Calling the whole thing an evil empire isn't going to make any progress: your attacks are easily fended off.

> If you want to attack you need to be able to identify where they are acting badly and strike at that weakness.

Can you give an example?

>> Even punishing them(fines) for their mistakes is not even a slap on their wrist at their scale we've seen this happening time and again

If this wasn't true then there wouldn't be sub-list for every wrongdoing spanning decades[1] i.e. If the punishments achieved it's purpose we wouldn't see continued offenses under same category.

[1] https://en.wikipedia.org/wiki/Criticism_of_Amazon

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#98

Post by unityByFreedom »

Abishek_Muthian wrote: post #97
unityByFreedom wrote: post #96

Earlier quoted context omitted.

They're a huge company. If you want to attack you need to be able to identify where they are acting badly and strike at that weakness. Calling the whole thing an evil empire isn't going to make any progress: your attacks are easily fended off.

> If you want to attack you need to be able to identify where they are acting badly and strike at that weakness. Can you give an example? >> Even punishing them(fines) for their mistakes is not even a slap on their wrist at their scale we've seen this happening time and again If this wasn't true then there wouldn't be sub-list for every wrongdoing spanning decades[1] i.e. If the punishments achieved it's purpose we w…

> Can you give an example?

Forcing employees to pee in bottles is pretty bad.

> If the punishments achieved it's purpose we wouldn't see continued offenses under same category.

Hah, the notion that punishments always work is funny. Good one. Try that on your kids. Look, making a punishment bigger may or may not help. I don't know about that. You can't just fine them out of existence for no reason, however. Bezos isn't Jack Ma and this isn't China.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#99

Post by Abishek_Muthian »

unityByFreedom wrote: post #98
Abishek_Muthian wrote: post #97

Earlier quoted context omitted.

> If you want to attack you need to be able to identify where they are acting badly and strike at that weakness. Can you give an example? >> Even punishing them(fines) for their mistakes is not even a slap on their wrist at their scale we've seen this happening time and again If this wasn't true then there wouldn't be sub-list for every wrongdoing spanning decades[1] i.e. If the punishments achieved it's purpose we w…

> Can you give an example? Forcing employees to pee in bottles is pretty bad. > If the punishments achieved it's purpose we wouldn't see continued offenses under same category. Hah, the notion that punishments always work is funny. Good one. Try that on your kids. Look, making a punishment bigger may or may not help. I don't know about that. You can't just fine them out of existence for no reason, however. Bezos isn'…

> Forcing employees to pee in bottles is pretty bad.

Yes it is(Like dozens of other things listed in that Wiki), But I didn't ask that. I wanted to know how do you suggest we strike at 'the weakness of Amazon' to fix it (Like you suggested) that it treats it's employees with dignity and respect after that?

Because IMO if there's anything, It could have been done when Amazon's mistreatment of its workers surfaced way back in 2001.

>Hah, the notion that punishments always work is funny. Good one. Try that on your kids.

That's the point of punishments, isn't it? Just calling them out is convenient but does nothing.

> Look, making a punishment bigger may or may not help.

That's the whole point. They're too big for any punishments, Hence qualified people have been arguing that private companies shouldn't get that big(It undermines the basic tenants of democracy reg accumulation of power); But unfortunately those who can take action about it are also getting big thanks to $AMZN.

> however. Bezos isn't Jack Ma and this isn't China.

>> Calling the whole thing an evil empire isn't going to make any progress: your attacks are easily fended off.

Re: Amazon Elasticsearch Service Is Now Amazon OpenSearch Service

#100

Post by unityByFreedom »

Abishek_Muthian wrote: post #99
unityByFreedom wrote: post #98

Earlier quoted context omitted.

> Can you give an example? Forcing employees to pee in bottles is pretty bad. > If the punishments achieved it's purpose we wouldn't see continued offenses under same category. Hah, the notion that punishments always work is funny. Good one. Try that on your kids. Look, making a punishment bigger may or may not help. I don't know about that. You can't just fine them out of existence for no reason, however. Bezos isn'…

> Forcing employees to pee in bottles is pretty bad. Yes it is(Like dozens of other things listed in that Wiki), But I didn't ask that. I wanted to know how do you suggest we strike at 'the weakness of Amazon' to fix it (Like you suggested) that it treats it's employees with dignity and respect after that? Because IMO if there's anything, It could have been done when Amazon's mistreatment of its workers surfaced way…

> I wanted to know how do you suggest we strike at 'the weakness of Amazon' to fix it (Like you suggested) that it treats it's employees with dignity and respect after that?

You call them out for the individual wrongs like those listed on the Wikipedia page. Vote with your wallet. Debate and discuss.

> They're too big for any punishments

I don't agree with that. No company survives forever.

> Jack Ma & China

The US is not going to fine Amazon out of business like the CCP did to Jack Ma. The CCP is evil, not the people. Their crackdown on people's ability to freely debate and discuss just shoots the country's most valuable asset, its people, in its own foot over in over in order to maintain footholds for a relatively small group controlling the party. Any creative ideas are squashed. If the CCP would let up they would give rise to a new generation of ingenuity in China. Of course some CCP members might face criticism for that. It's up to them to decide whether or not to move in that direction. Nobody likes to be on the receiving end of criticism and you need to be strong to take it.

Post reply on HN