Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenSearch: AWS fork of Elasticsearch and Kibana

aws.amazon.com

Post reply on HN
Print view View original

131–140 of 438 posts

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#131

Post by manigandham »

fallat wrote: post #84

So as someone who has heard about Elasticsearch for years and years, and seen all this, right this moment I've decided to see what it really is. On their home page, "Why use Elastic search?", the reasons are basically: * It's fast! * It does a lot of stuff! * It has some tools to visualize data! * It's distributed!! I have to say this is not very appealing to me since it sounds like something any database could do.

The Elastic website has a dedicated page to explain: https://www.elastic.co/what-is/elasticsearch

"Elasticsearch is a distributed, free and open search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. Elasticsearch is built on Apache Lucene and was first released in 2010..."

I suggest understanding what it is first before comparing it to other databases.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#132

Post by x3n0ph3n3 »

damon_c wrote: post #106
busterarm wrote: post #59

Earlier quoted context omitted.

Indeed, although I'd say for a majority of situations, Amazon's hosted Elasticsearch service is a complete non-starter.

Why is it a non-starter? I have a few ElasticSearch clusters that have been running for years with zero effort/downtime/hassle.

It's pretty bad when you reach the 40-50 node scale with 10's or 100's of TB of data. I've had about half dozen calls with their service team about this over the last year.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#133

Post by manigandham »

TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

People want to pay for services, not software and licenses. They want turn-key solutions that are available via API and GUI, instantly and on-demand. This is the fundamental reason why AWS is so successful and the demand is constantly proven with every new product launch.

Elastic (and other vendors) complaining about this instead of using it for their own success is a problem of their own making. At least a few companies are finally learning.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#134

Post by vio2 »

calmoo wrote: post #107
shankspeaks wrote: post #96

Imagine if they went after Mongo next? Atlas is a virtual monopoly for Mongo solely due to SSPL, and it has created a ridiculously overpriced ecosystem for hosted and managed services, and tooling around it. Parking the technical merits to one side, considering the sheer number of devs and early-stage products that are built on Mongo, I'd love for someone to go after them next.

Amazon already have DocumentDB which clones the Mongo API. I don't think its forked though, they just use a barely mongo compatible wrapper around their own db engine.

It's not nearly as compatible as you might think. Interestingly enough, MongoDB's CTO managed RDS at AWS.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#135

Post by reflexe »

TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

It it quite funny, elasticsearch is also kind of wrapping an open source library (Apache Lucene) and selling it as their own product.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#136

Post by g9yuayon »

hintymad wrote: post #94
TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

It sounds trivial to "wrap open source software", but surprisingly it is big value-add to thousands of companies. We can't just look at successful companies like Netflix to downplay the challenges of operating a service. Not every company knows how to operate complex systems under manageable cost. How many companies can really manage a Kafka cluster, let alone scaling it, for instance? Indeed, even companies that peo…

A key reason for Netflix to have an easy-to-operate infrastructure is that Netflix prioritizes productivity and scalability. They specifically did three things:

1. No fixed deadline, with a few exceptions of course, for platform-related projects.

2. Promotion/salary negotiation was not tied directly to release of external features.

3. A single engineer could be responsible for more than one service for the entire company, with 24x7 oncall.

With Netflix establishing such incentives, engineers naturally focus on getting infrastructure right, to the point that oncall 24x7 is a non-issue.

So, yeah, culture matters, big time.

Edit: another incentive was that a service was measured by its adoption. The more people praised it, the more successful the service would be. Requiring meetings to get buy-in for a new service was considered a sign of potential failure. As a result, every single team focused on making the value proposition of their services obvious. Path of least resistance was a given instead of a debated topic.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#137

Post by sdesol »

binarymax wrote: post #124
sdesol wrote: post #78

Will be interesting to see the resources that AWS will throw at this. You can get a sense of the resource that elastic.co is throwing at elasticsearch at https://public-001.gitsense.com/insights/github/repos?r=gith... I'm currently indexing the fork, so in about an hour or two, I'll provide the insights for the fork as well.

Elastic spends so much time and effort on making sure that their search is performant (and they are not shy about deprecating and removing features that are slow). I think this is where Elastic will continue to shine. It's one thing to add features, it's another to make it so they work well and make sure the integrating product team doesn't shoot themselves in the foot.

What may hurt them though, is the number of customers that currently feel things are currently "good enough". I don't know what their sales engagement looks like, so I'm not sure if this will really hurt them or not.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#138

Post by scarface74 »

TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

There is absolutely nothing “cloud agnostic” about using Terraform. Every provisioner is specific to a cloud provider. If you are at any scale, moving a k8s cluster is the least of your issues.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#139

Post by manigandham »

maximente wrote: post #46
TechBro8615 wrote: post #21

Earlier quoted context omitted.

It's absolutely price gouging. I'm not going to rant about this for the 100th time, but at least I'm in good company [0]. Do the math on the cost you pay if you saturate 1gbps for a month vs. the cost you pay for 1gbps IP transit at basically any colocation provider. Really this is the secret sauce of the cloud. Create new abstraction layers where you can charge for logical separation on a physical basis. First VMs,…

boy howdy, with all the flak i hear about this and the awesome talent in tech, you'd figure an entrepreneur or 1000 would take a stab at this, make it better, charge less. apparently there's gazillions to be made by even charging 50 percent of what AWS does. so, when should we expect this gloriously efficient competitive market to kick in to action? my guess is that the AWS ecosystem, despite "price gouging", is simp…

You're replying to a comment that includes a tweet from the CEO of Cloudflare, which is quite literally providing that competition with free bandwidth and an increasing suite of computing products.

There are plenty of other platforms as well, like Digitalocean, that have much lower bandwidth pricing.

Re: OpenSearch: AWS fork of Elasticsearch and Kibana

#140

Post by gatvol »

minhazm wrote: post #13
TechBro8615 wrote: post #9

It's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / cons…

Is it really price gouging for bandwidth? Or is bandwidth just really expensive in general? I honestly don't know. I would assume if it was actually much cheaper one of the cloud's would undercut the other to get customers.

If you think AWS is expensive, give AZURE a go and be appalled.
Post reply on HN