Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elasticsearch 1.0.0 released

elasticsearch.org

Post reply on HN
Print view View original

51–60 of 139 posts

Re: Elasticsearch 1.0.0 released

#51

Post by nzadrozny »

brasetvik wrote: post #50
RyanZAG wrote: post #36

Earlier quoted context omitted.

Complexity. Having two copies of the data means more dev time, more resources required to shift the data around, etc. Having just 1 data store that can also handle all your searching is like the holy grail. As you say, not sure if Solr/ES/Lucene are there yet - but they're definitely very very close. There is no theoretical barrier either - it just comes down to closing bugs, and the ES/Lucene team are very good at c…

Sometimes, I actually find it easier to have more systems that do their job really well and sync things between them, rather than trying to get a single system to do everything. For example, Postgres lets you reason about integrity, atomicity and transactional boundaries, and whether things are really safely stored with synchronous replication. If Postgres returns after a commit, I trust it. However, that requires me…

I'd like to point out that two competing founders of hosted Elasticsearch as a service agree: ES is great, but not a general-purpose data store :-)

Re: Elasticsearch 1.0.0 released

#52

Post by alisson »

polyfractal wrote: post #45
alisson wrote: post #44

Earlier quoted context omitted.

On ElasticSearch you have to update the whole document, no commands to manipulate them. You don't have commands like: $set, $addToSet, $pop, etc.. You need to have a good understanding of how tokenizers and analyzers work to be able to create good results for your data. I have difficulties matching documents with the exact title being searched for. On MongoDB that just works, on ElasticSearch you need to configure it…

Regarding updates, you can use the Update API for partial updates, and include a script to do things like "counter += 1" or "add value to existing array" . Internally it is still reindexing the entire document, but from your application's perspective, the Update API is a lot friendlier. http://www.elasticsearch.org/guide/en/elasticsearch/referenc...

Thanks for pointing that out, it will be really useful!

Re: Elasticsearch 1.0.0 released

#53

Post by tracker1 »

RyanZAG wrote: post #7

Elasticsearch is really awesome for searching, but what most people don't realize is that it makes a better MongoDB than MongoDB while giving you that searching too.

I would suggest that everyone who is considering one, look at both... When I looked into both, about a year and a half ago, I found that geospacial searches worked better in MongoDB at the time, and shaping my data to fit was more awkward with ElasticSearch.

That said, it's definitely worth looking into both, depending on what your needs are.

Re: Elasticsearch 1.0.0 released

#55

Post by room271 »

camus2 wrote: post #26
kainosnoema wrote: post #10

Earlier quoted context omitted.

I'm surprised so many people miss this. Out of the box, Elasticsearch is a distributed NoSQL store with better write consistency (and arguably performance) than MongoDB offers in its default configuration. The major missing feature was backup snapshots and restores, which 1.0 delivers—along with aggregations that more than rival MongoDBs. The team has intentionally avoided marketing themselves as a NoSQL store (was t…

did not know all that stuff, could Elasticsearch be the holy grail of document stores ?

No. The choice of datastore is still incredibly complicated in the distributed world; it's all about tradeoffs really.

For example, Elasticsearch has poor availability characteristics - both because it is master-slave and because it focuses on ensuring consistency - relative to, for example, something like Riak.

Re: Elasticsearch 1.0.0 released

#56

Post by curun1r »

brasetvik wrote: post #21
axefrog wrote: post #18

Earlier quoted context omitted.

What limitations should one be aware of that would make ElasticSearch not a viable candidate where something like MongoDB would be a better fit?

I can't comment much on MongoDB, but I've written a bit things to keep in mind when considering Elasticsearch as a NoSQL store here: https://www.found.no/foundation/elasticsearch-as-nosql/

An interesting read, but I'd disagree with your contention that NoSQL isn't about ACID. When NoSQL databases started coming out, it was really about which CAP guarantee a database chooses to compromise. Traditional SQL databases are either partition-intolerant or become unavailable (for writes) in the event of a partition. NoSQL databases compromise on consistency. If a database is claiming to be NoSQL and have ACID transactions, they've either disproven CAP or aren't part of the new group of distributed, partition-tolerant databases that people have been calling NoSQL. It's been said for a while that NoSQL is a terrible name for that group of technologies and now that we're getting databases with a non-SQL interface but also having consistency guarantees, the name is starting to cause even more confusion.

Side note: Happy Found customer here...you guys have made it much easier to run our ES index!

Re: Elasticsearch 1.0.0 released

#57

Post by tracker1 »

camus2 wrote: post #26
kainosnoema wrote: post #10

Earlier quoted context omitted.

I'm surprised so many people miss this. Out of the box, Elasticsearch is a distributed NoSQL store with better write consistency (and arguably performance) than MongoDB offers in its default configuration. The major missing feature was backup snapshots and restores, which 1.0 delivers—along with aggregations that more than rival MongoDBs. The team has intentionally avoided marketing themselves as a NoSQL store (was t…

did not know all that stuff, could Elasticsearch be the holy grail of document stores ?

It is definitely a nice, and flexible option.. it truly depends on what your needs are... If you're often updating parts of a document, MongoDB or RethinkDB may be better options. If you want integration where a lot of parts are SQL with some document ability, PostgreSQL + V8 is pretty compelling. Also, something like Cassandra may suit your needs better if you want a better and more predictable growth curve.

There's no holy grail of data storage... ElasticSearch is really nice, and if it fits your needs, more power to you.

Re: Elasticsearch 1.0.0 released

#58

Post by AznHisoka »

ddorian43 wrote: post #20
axefrog wrote: post #18

Earlier quoted context omitted.

What limitations should one be aware of that would make ElasticSearch not a viable candidate where something like MongoDB would be a better fit?

presharding You create a number of shards for each index(database) that you can't later expand.

What I've done, and I'm not totally sure if it's a best practice is I've over-allocated the # of shards. So if I think I need 5 shards, I create 50 or 100 shards instead. Then I'll have some app logic to determine the shard a document should go to. Initially all docs will go to shard 0. Then when that's full (around 15 GB of size, depends on your RAM), then I set all docs to go to shard 1. Of course, you'll need to be careful as you dun want duplicate documents in different shards.

The benefit of this is the as your app scales, you'll search only the shards needed. So if you have just 1 shard w/ data, u can tell ElasticSearch to just search in that 1 shard.

Re: Elasticsearch 1.0.0 released

#59

Post by dclara »

NDizzle wrote: post #19

I also took a few days a few weeks ago to setup elastic search after my mysql full text search fell apart. What I'm doing is slamming the full text output of OCRed PDFs into a MyISAM table, the entire document in a text field. What I'm afraid I'm not doing right is creating the web interface to search elasticsearch. What I'm using filters with the query string syntax[1] in the search box, pointing directly at that fu…

May I ask what you meant about "web based search implementation using elasticsearch"?

Do you mean that you use ES to do indexing on the backend of your documents and make it available on the web? Or do you mean that you use ES to index documents available on the web and let people to search for them?

Re: Elasticsearch 1.0.0 released

#60

Post by brasetvik »

nzadrozny wrote: post #51
brasetvik wrote: post #50

Earlier quoted context omitted.

Sometimes, I actually find it easier to have more systems that do their job really well and sync things between them, rather than trying to get a single system to do everything. For example, Postgres lets you reason about integrity, atomicity and transactional boundaries, and whether things are really safely stored with synchronous replication. If Postgres returns after a commit, I trust it. However, that requires me…

I'd like to point out that two competing founders of hosted Elasticsearch as a service agree: ES is great, but not a general-purpose data store :-)

Hi, Nick. :)

Indeed!

That said, it's great that more people are picking up Elasticsearch for new exciting things.

Elasticsearch has really pushed what constitutes a "search problem", and deserves lots of kudos for that! :)

Post reply on HN