Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenSearch: AWS fork of Elasticsearch and Kibana

aws.amazon.com

Post reply on HN
Print view View original

141–150 of 438 posts

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#141

Post by elric »

mgr86 wrote: post #71

Naive aside, but why would I want to use ElasticSearch or OpenSearch over Solr? Are ES and Solr not both based on Lucene?

In all three cases, Lucene is used as a "low level" (Java) API which provides search capabilities. OS, ES and Solr turn Lucene into a server, with features like horizontal scaling (ES Cluster, Solr Cloud). The major differences are in how well that all works, how easy it is to administer, how much caching and optimization is done on top of Lucene, etc.

I haven't extensively used ES, but I've used Solr a lot (and contributed to), and I can say that it's a mess. The community is not one of the better ones I've seen. Bugs and stability issues are often ignored. Patches sit around gathering dust. There are some gems and very clever people in the community, of course, but it seems like there are too few of them to cope with the large beast that Solr has become. If I were starting a new project in the search space, Solr would not make my shortlist.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#142

Post by kodah »

aardvarkr wrote: post #52
015a wrote: post #43

Earlier quoted context omitted.

One thing which surprised me: Elastic has a market capitalization of ~$11B. I think that changes some of the more floaty ethical concerns. This is not a David vs Goliath situation. This is Goliath vs Super-Goliath. At this point, I'm much less interested in the drama of which mega-corp is screwing over the other. I'm more interested in: how does it affect me? When the titans are done trampling over the rest of us, wh…

Uhmmm I’m pretty sure David vs Goliath is talking about scale between competition. Saying that $11B is Goliath just because you’re sitting at $1M doesn’t mean they’re not in a crazy mismatched fight against a $2T company. In the same way you could be in a David vs Goliath situation yourself if you with $1M in wealth tried to sue someone with $25K of wealth. Everything is relative. Doesn’t mean it’s not a crazy unfair…

Put another way, the $11B Goliath is .55% of Super Goliaths size.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#143

Post by bmcahren »

I feel Amazon took the feedback from the DocumentDB/MongoDB fiasco to heart and made positive change in their approach.

DocumentDB is a closed source proprietary database created by Amazon to emulate the MongoDB API. Think Google's Dalvik runtime vs Sun/Oracle's JVM.

This time around we have an open source fork of ES with big backers all contributing and very permissive licensing.

In both cases, Amazon gets to implement AWS-specific upgrades to management to depend heavily on EBS replication rather than application-layer replication. Would it be nice to have that secret sauce that makes Aurora/DocumentDB so nice to use compared to self-hosting or RDS? Of course. Do we have to have it to consider using or contributing to the open source software? No.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#145

Post by VectorLock »

minhazm wrote: post #13
TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

Is it really price gouging for bandwidth? Or is bandwidth just really expensive in general? I honestly don't know. I would assume if it was actually much cheaper one of the cloud's would undercut the other to get customers.

I'm kind of surprised that people are this upset about how much AWS charges for bandwidth. They may charge more for bandwidth than a colo would but they're not a colo. A colo you get a network port and -thats it- you provide everything else yourself, with its attendant cost, and you roll that up into your total bill.

If a colo provides you a 1 Gbps connection if you use less, you don't get a refund. And most of the time you don't get 24/7 saturation, or you get charged on some 95th percentile billing, and their networks are almost always oversubscribed anyways.

AWS is trying to disincentivize using it as a dumb pipe. They want you to use it smartly and if you just want to push static data there are much more cost effective ways to do it, such as CDNs, which are more cost effective for both you AND AWS.

Comparing AWS bandwidth costs and Colo and even other clouds like Oracle isn't fair because different things are associated with that cost.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#146

Post by VectorLock »

toast0 wrote: post #74
maximente wrote: post #46

Earlier quoted context omitted.

boy howdy, with all the flak i hear about this and the awesome talent in tech, you'd figure an entrepreneur or 1000 would take a stab at this, make it better, charge less. apparently there's gazillions to be made by even charging 50 percent of what AWS does. so, when should we expect this gloriously efficient competitive market to kick in to action? my guess is that the AWS ecosystem, despite "price gouging", is simp…

There's tons of businesses that are happy to charge less for bandwidth; so it's clear Amazon (and some of the other high tier cloud services) are overcharging on this maybe by a factor of 10, although since transit bits are not all equal, someone with more detail could make a case that the overcharging is less. It's easyish to compete on bandwidth costs, but Amazon has a lot of other features many people want; it's h…

This is like going to a fancy restaurant and being upset that they charge so much for a steak when you can get beef at the supermarket for much less.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#147

Post by JamesSwift »

fastball wrote: post #117
JamesSwift wrote: post #23

Earlier quoted context omitted.

The lock in is real, but they are also the best cloud option and its not even close in my opinion.

What would you say they lock you into? What is AWS doing specifically that makes it harder to move between providers than if they were not doing that thing?

1) A vast majority of users are relying on managed services to some degree.

2) Permissions as a whole are usually very ingrained into much of what orgs rely on.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#148

Post by darkwater »

paxys wrote: post #105
brainless wrote: post #82

Earlier quoted context omitted.

The time when Apache or MySQL started out was very different. Imagine where the internet would be if cloud computing itself didn't take off. Do you remember a time when there were hundreds of hosting providers? Do you remember WebHostingTalk where admins would go to check hosting offers from suppliers around the world? The monopolies finished that era. So I don't think that software companies trying to adapt now can…

There are more hosting providers today than in the era you are talking about. AWS has a "monopoly" simply because large companies are using it, and back in the day those companies would have run their own datacenters not used a shared PHP host. For a personal site or startup you have a thousand other options.

I'm not even that sure this is true in absolute numbers, but I'm pretty sure it is not in relative figures. There are waaaay more customer/businesses on the Internet today than there were 15 years ago and not that many providers more.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#149

Post by stavros »

throwaway823882 wrote: post #95
TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

> Is Amazon too big and a bully? Is Starbucks too big and a bully? Sure, they force Mom and Pop shops to close by out-competing them, and that sucks. But bullying? I believe that's just Capitalism. Or say you're 6'6 and weigh 230LBS and you join a football team full of people who are 5'9 and 175lbs. Are you a bully just because you're bigger? ES basically handed them a platter with a goose laying golden eggs and a si…

> Is Starbucks too big and a bully? Sure, they force Mom and Pop shops to close by out-competing them, and that sucks. But bullying? I believe that's just Capitalism.

I'd say it's more like a mom-and-pop coffee shop giving free coffee to patrons hoping to make money on cookies, and Starbucks coming in, taking the free coffee, opening a nice stand right next to the shop and selling the coffee they got for free.

I'm having trouble justifying this ethically.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#150

Post by mumblemumble »

It's been said that the best way to fortify your business is to use your clout to make the world inhospitable for adjacent businesses.

As someone who is not currently in the cloud, that idea strikes me as being very pertinent to what's happening here. Increasingly many technologies are becoming cloud-only, or have non-cloud offerings that are decidedly second-class. Elastic offers on-prem support. I doubt Amazon will be doing the same with OpenSearch.

It may be a subtle effect, but it's pushing the world in a direction that makes me uncomfortable. If it's harder for non-cloud-based companies to maintain non-cloud-based offerings, then that will push the industry even more toward being dominated by SaaS products. And these products often leave clients and users locked in, with limited control over their own data, and, by extension, reduced ability to control their own fates. What I worry about is that we may be witnessing a return of Embrace, Extend, Extinguish, only in a new form that's even more dangerous because it's harder to see.

I appreciate the discomfort people have about the SSPL. It is a departure from the original ideas behind FOSS. But, at least as I see it, those open source principles were never an end in and of themselves. They're a means to a greater end: digital autonomy. To the extent that very large companies seem to be learning how to co-opt FOSS in order to re-assert control, FOSS's ability, in its current incarnation, to serve that end may be waning.

Post reply on HN