Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elasticsearch 1.0.0 released

elasticsearch.org

Post reply on HN
Print view View original

101–110 of 139 posts

Re: Elasticsearch 1.0.0 released

#101

Post by nzadrozny »

wikyd wrote: post #100
nzadrozny wrote: post #29

Earlier quoted context omitted.

Ditto open file handles, which is easy to push when aggressively over-sharding. Not an uncommon mistake for the enthusiastic newbie. Having supported Solr/ES/Lucene in production for 4+ years now (websolr.com / bonsai.io) I would be pretty hesitant to trust Lucene in general as a primary data store. Beautiful for secondary indexing, but otherwise, Why Not Postgres?™ ;)

I think the CSS for bonsai.io is not loading.

Thanks. I screwed up our CDN settings while trying to push out some changes. Working on that :-)

Re: Elasticsearch 1.0.0 released

#103

Post by vhost- »

I'd be curious to see how well Elastic Search holds up to Endeca. I'm currently stuck maintaining some Endeca instances and it's a nightmare. I wish I could go back to ES.

At my last place of work, ES was beautiful and required little work to get a very fast, workable search in place.

Re: Elasticsearch 1.0.0 released

#104

Post by aquadrop »

AznHisoka wrote: post #58
ddorian43 wrote: post #20

Earlier quoted context omitted.

presharding You create a number of shards for each index(database) that you can't later expand.

What I've done, and I'm not totally sure if it's a best practice is I've over-allocated the # of shards. So if I think I need 5 shards, I create 50 or 100 shards instead. Then I'll have some app logic to determine the shard a document should go to. Initially all docs will go to shard 0. Then when that's full (around 15 GB of size, depends on your RAM), then I set all docs to go to shard 1. Of course, you'll need to b…

So, what happens when you fill up the last shard?

Re: Elasticsearch 1.0.0 released

#105

Post by pyotrgalois »

Great news. In every new project that we create (in general REST JSON APIs made with nodejs, erlang or rails that are consumed by iOS and android clients) we always finish using postgresql, redis and elasticsearch. Great tools.

Re: Elasticsearch 1.0.0 released

#106

Post by zcrar70 »

mavelikara wrote: post #69

ES seems to have ability to run analytic queries. I have read about people using it as an OLAP solution [1], although I have not yet read anyone describe their experience. In that respect how does ES analytics capabilities compare against: 1) Dremel clones [2] like Impala & Presto (for near real-time, ad hoc analytic queries over large datasets) 2) Lambda Architecture [3] systems (where queries are known up- front, b…

I would also be interested in this.

Re: Elasticsearch 1.0.0 released

#107

Post by spooneybarger »

gibrown wrote: post #23
kainosnoema wrote: post #10

Earlier quoted context omitted.

I'm surprised so many people miss this. Out of the box, Elasticsearch is a distributed NoSQL store with better write consistency (and arguably performance) than MongoDB offers in its default configuration. The major missing feature was backup snapshots and restores, which 1.0 delivers—along with aggregations that more than rival MongoDBs. The team has intentionally avoided marketing themselves as a NoSQL store (was t…

While I agree with the sentiment, I think Shay (lead ES developer) has explicitly said that he does not consider ES to be a data store... yet. I think this is mostly due to maturity. I help run a large ES cluster (with canonical data in MySQL), and I consider this cautious attitude by the ES developers to be a good thing.

He has indeed said that. We hosted the Elasticsearch meetup in NYC a couple weeks ago and specifically said it.

Re: Elasticsearch 1.0.0 released

#108

Post by AznHisoka »

bilbo0s wrote: post #14
RyanZAG wrote: post #7

Elasticsearch is really awesome for searching, but what most people don't realize is that it makes a better MongoDB than MongoDB while giving you that searching too.

This. A THOUSAND TIMES "This". The one drawback ES had in the bad old days was that backup and restore was a nightmare... ESPECIALLY on AWS. The new system they introduced was so simple I was concerned about updating to it because I was SURE something would go south. But it all just worked. I still have the Couch to ES replication running because I'm anal like that... but really... yeah... you can do without Couchbas…

Just curious, if I'm using say version 0.92, how would I go about backing up my ElasticSearch instance. Besides creating a replica in a server, then "freezing" it by disconnecting the server?

Re: Elasticsearch 1.0.0 released

#109

Post by polyfractal »

AznHisoka wrote: post #108
bilbo0s wrote: post #14

Earlier quoted context omitted.

This. A THOUSAND TIMES "This". The one drawback ES had in the bad old days was that backup and restore was a nightmare... ESPECIALLY on AWS. The new system they introduced was so simple I was concerned about updating to it because I was SURE something would go south. But it all just worked. I still have the Couch to ES replication running because I'm anal like that... but really... yeah... you can do without Couchbas…

Just curious, if I'm using say version 0.92, how would I go about backing up my ElasticSearch instance. Besides creating a replica in a server, then "freezing" it by disconnecting the server?

The pre-Snapshot/restore method is:

- Pause indexing

- Issue a flush request

- Rsync data directories somewhere

- Resume indexing

This is technically a very naive approach, since a simple rsync of the data dirs will include replicas too. If you were more diligent you could check the state files in each shard directory and only copy out the primaries.

Re: Elasticsearch 1.0.0 released

#110

Post by roryokane »

sandstrom wrote: post #74

This gem is from the 'breaking changes' list: “Geo queries used to use miles as the default unit. And we all know what happened at NASA because of that decision. The new default unit is meters.” I like this release already.

Link to that page: http://www.elasticsearch.org/guide/en/elasticsearch/referenc...
Post reply on HN