Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Major Changes from Solr 4 to Solr 5

cwiki.apache.org

Post reply on HN
Print view View original

21–30 of 47 posts

Re: Major Changes from Solr 4 to Solr 5

#21

Post by lobster_johnson »

shironinja wrote: post #17
sagivo wrote: post #4

Any reasons to prefer Solr over elastic-search?

For your use case set up both and then do some testing. Speaking for myself I did that and found that SOLR was a lot more performant. I needed a high-traffic solution without a bunch of servers. I find that ES tries to do too much with all the dashboards and monitoring etc. SOLR keeps it simple and thereby does not incur the performance penalty. Also when an ES cluster goes south (cluster health "yellow" or "red") it…

Does Solr have the same problem as ES where you can't modify an index mapping after creating it? With ES you have to reindex everything into a new temporary index and then swap the new and old indexes. It's a terrible design, especially considering that ES already has all the original data and should be perfectly capable of doing it itself, incrementally.

Re: Major Changes from Solr 4 to Solr 5

#22

Post by BonoboBoner »

Tharkun wrote: post #5
thinkcomp wrote: post #3

Does Solr 5.0 support password-protecting the admin interface yet without spending hours trying to wrangle custom XML files? It seems like a pretty basic requirement for a web-based application. I've tried things like this http://community.zimbra.com/documentation/w/documentation/se... repeatedly. They never seem to work right.

Solr is not a web-based application. You shouldn't directly expose your Solr instance to anyone. Regardless of whether or not you secure your admin interface. That's not Solr's core business, and I don't see why they should was their efforts on it. Have Solr listen on localhost and have your web app talk to Solr. If your Solr is visible to the world, you're doing it wrong. Edit: by saying that it's not a web-based ap…

While you are technically right, that kind of thinking gave people access to a ton of MongoDB databases last week.

Re: Major Changes from Solr 4 to Solr 5

#23

Post by shironinja »

lobster_johnson wrote: post #21
shironinja wrote: post #17

Earlier quoted context omitted.

For your use case set up both and then do some testing. Speaking for myself I did that and found that SOLR was a lot more performant. I needed a high-traffic solution without a bunch of servers. I find that ES tries to do too much with all the dashboards and monitoring etc. SOLR keeps it simple and thereby does not incur the performance penalty. Also when an ES cluster goes south (cluster health "yellow" or "red") it…

Does Solr have the same problem as ES where you can't modify an index mapping after creating it? With ES you have to reindex everything into a new temporary index and then swap the new and old indexes. It's a terrible design, especially considering that ES already has all the original data and should be perfectly capable of doing it itself, incrementally.

It is similar in SOLR (and other NoSQL data stores) currently .. at least as far as I am aware. It can be quite intensive especially since ES, SOLR, etc like to use as much memory as they can for fast access. Any non-trivial application should have duplicate servers / instances to soak up the traffic while a change like that is migrated.

Re: Major Changes from Solr 4 to Solr 5

#24

Post by arafalov »

sagivo wrote: post #4

Any reasons to prefer Solr over elastic-search?

Solr is a lot more configurable (Search Components, Update Request Processors, configurable Schemaless processing sequence, etc).

Solr includes Admin UI console which is free for production, ES has one that's only free for development.

Solr has contributors from a lot more different companies, so grows into multiple directions at once.

If you want to compare on a technical level, you can see my presentation from the Lucene/Solr Revolution back in November: http://www.slideshare.net/arafalov/solr-vs-elasticsearch-cas...

Solr is completely free. If that's not an issue for you and you are ready to pay, then you should compare Elasticsearch to LucidWorks Fusion, not directly to Solr.

Re: Major Changes from Solr 4 to Solr 5

#25

Post by arafalov »

lobster_johnson wrote: post #21
shironinja wrote: post #17

Earlier quoted context omitted.

For your use case set up both and then do some testing. Speaking for myself I did that and found that SOLR was a lot more performant. I needed a high-traffic solution without a bunch of servers. I find that ES tries to do too much with all the dashboards and monitoring etc. SOLR keeps it simple and thereby does not incur the performance penalty. Also when an ES cluster goes south (cluster health "yellow" or "red") it…

Does Solr have the same problem as ES where you can't modify an index mapping after creating it? With ES you have to reindex everything into a new temporary index and then swap the new and old indexes. It's a terrible design, especially considering that ES already has all the original data and should be perfectly capable of doing it itself, incrementally.

The primary limitation comes from Lucene, which powers both. With Solr, you could change the definition and reload the core, but you will be getting some weird artifacts for the previously-indexed content.

Neither Solr nor Elasticsearch should be treated as a primary data store, so that's probably why reindex-in-place is not the highest priority.

Re: Major Changes from Solr 4 to Solr 5

#26

Post by jonbaer »

thinkcomp wrote: post #3

Does Solr 5.0 support password-protecting the admin interface yet without spending hours trying to wrangle custom XML files? It seems like a pretty basic requirement for a web-based application. I've tried things like this http://community.zimbra.com/documentation/w/documentation/se... repeatedly. They never seem to work right.

If you need a solid easy-to-config security wrapped around Solr I would recommend taking a peek @ Fusion from Lucidworks, http://lucidworks.com/product/solr-enterprise/

Re: Major Changes from Solr 4 to Solr 5

#27

Post by gizmogwai »

lobster_johnson wrote: post #21
shironinja wrote: post #17

Earlier quoted context omitted.

For your use case set up both and then do some testing. Speaking for myself I did that and found that SOLR was a lot more performant. I needed a high-traffic solution without a bunch of servers. I find that ES tries to do too much with all the dashboards and monitoring etc. SOLR keeps it simple and thereby does not incur the performance penalty. Also when an ES cluster goes south (cluster health "yellow" or "red") it…

Does Solr have the same problem as ES where you can't modify an index mapping after creating it? With ES you have to reindex everything into a new temporary index and then swap the new and old indexes. It's a terrible design, especially considering that ES already has all the original data and should be perfectly capable of doing it itself, incrementally.

No, they don't. Behind the scene, it's Lucene that is creating a set of inverted indexes. While extremely performant for text search, it is also destructive and partial comparing to the initial dataset. So, there is no way to migrate an index to another one.

Re: Major Changes from Solr 4 to Solr 5

#28

Post by lobster_johnson »

arafalov wrote: post #25
lobster_johnson wrote: post #21

Earlier quoted context omitted.

Does Solr have the same problem as ES where you can't modify an index mapping after creating it? With ES you have to reindex everything into a new temporary index and then swap the new and old indexes. It's a terrible design, especially considering that ES already has all the original data and should be perfectly capable of doing it itself, incrementally.

The primary limitation comes from Lucene, which powers both. With Solr, you could change the definition and reload the core, but you will be getting some weird artifacts for the previously-indexed content. Neither Solr nor Elasticsearch should be treated as a primary data store, so that's probably why reindex-in-place is not the highest priority.

I think treating Solr/ES as a primary data store would be a horrible idea. But the problem is equally annoying if you're using it as an expendable search index.

There is no technical reason why Solr/ES could not do diff-based indexing. ES (I don't know about Solr) admittedly uses a single Lucene index per logical index, so changing a single field mapping involves reindexing the whole index, not just that one mapping.

But if the mappings were properly versioned ES could simply create a new version, index everything (from the original contents), and then swap. Locking the original index should be a non-issue.

Re: Major Changes from Solr 4 to Solr 5

#29

Post by lobster_johnson »

gizmogwai wrote: post #27
lobster_johnson wrote: post #21

Earlier quoted context omitted.

Does Solr have the same problem as ES where you can't modify an index mapping after creating it? With ES you have to reindex everything into a new temporary index and then swap the new and old indexes. It's a terrible design, especially considering that ES already has all the original data and should be perfectly capable of doing it itself, incrementally.

No, they don't. Behind the scene, it's Lucene that is creating a set of inverted indexes. While extremely performant for text search, it is also destructive and partial comparing to the initial dataset. So, there is no way to migrate an index to another one.

ES, by default, stores the original document in the nested "_source" document, which could be used to reindex data from scratch.

Re: Major Changes from Solr 4 to Solr 5

#30

Post by TallGuyShort »

capkutay wrote: post #20
chatman wrote: post #6

Earlier quoted context omitted.

Solr is backed by the Apache community while ES is backed by a private entity. Is that a good reason?

I don't know what you mean when you say backed by the Apache Community, but Elasticsearch is also an open source, apache-licensed project that has a commercial, private counterpart. I think every major open source project has this now (Hadoop has Cloudera/Hortonworks/MapR, Spark has databricks).

>> I don't know what you mean when you say backed by the Apache Community

It's a matter of who does the releasing (i.e. curating of patches, responsibility for building community, etc.): Solr is governed by the Apache Software Foundation and associated volunteers, ElasticSearch is governed by the private entity.

IMO (and this is an increasingly rare opinion) the license is the most important piece and they're both under the ASL 2.0.

Post reply on HN