ES has far more use than devops - most common usecase ive seen is tagged document searching
Ask HN: Are Lucene/Solr/ES Still Used for Search?
71–80 of 223 posts
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#72We use ES heavily. Most of our queries are basic document filtering plus some geospatial stuff. We could probably have done it with Postgres/PostGIS, but with AWS manages ES, it's all "good enough" -- we can do geospatial searches on millions of documents with response times around 100ms. The other part I like about ES is how it's easy to scale out across machines, which lets us handle quite a bit of load and tolerate failures easily. We have a cluster of 5 m4.large instances and it only runs us about $600/mo. Like others have said, tuning AWS ES sucks, but it's always been good enough for us.
We've run into some pain points like trying to index very large shapes into a geospatial index, but have workarounds for basically everything now. We also had a problem where when AWS had the outage around autoscaling groups a few months ago, we lost 3/5 of our instances and had to reindex some data from backups. That was the worst thing that's happened.
I'm sure there would be better/faster/cheaper ways of doing what we do, but for what we get out of the box for the price, it's going to take a lot for us to move away from it for now.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#73Yes but the search market has shifted.
We're at a point where Lucene and family are used for increasingly sophisticated use cases. The commodity end of the market used to be dominated by open source (Solr connected to Drupal for example)
Now for commodity sites there’s so many SaaS search products it doesn’t make as much sense to hook up Solr or ES to make your blog or university website or whatever searchable. A lot of basic search use cases are covered by products where you don’t want to have to hire a team to manage search.
But at the higher end apps with search, customization, especially of doing domain specific relevance at scale, is often a product differentiator (but often not so important or weird you should write your own engine). So this is where these systems thrive...
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#74The company I'm working is using Solr. We collect the data by either paying, for free or by using crawlers. Then we query the data from Solr if it is not a meta data. And there are other types of databases too that we use.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#75Within my current organisation, two main search options are Solr and Elasticsearch, both based on Lucene.
Generalising a bit, Solr is more targetted at enterprise search and unstructured content search (e.g. bundled with most content management systems), and Elasticsearch is more targetted at data analytics and structured data search (e.g. with the ELK stack). Again simplifying a bit, Solr can be a bit more configurable and/or work better with the types of data that benefit from more configuration, and Elasticsearch can work better with the types of data that work more "out of the box".
I'd agree that there don't seem to be a huge number of openings for specialist search roles or a huge number of people specialising in search, but it is often part of another role and there are often people who have touched on search in their roles. That suggests that many people are just using it with largely default setups. Having said that, things like advanced relevancy tuning, if you need it, is a very much under appreciated skillset, and definitely needs someone with good experience or ability to learn.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#76Yes.
Also, just recently MongoDB v4.2 added Lucene as an embedded engine for text search capabilities
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#77We use ElasticSearch at Lawmatics, and it powers more functionality than just our search! We use it to power our Automation targeting engine, reporting features, audience builder and pagination, filtering, sorting of data tables.
We denormalize associated records into one Index. And any record that we need to find based on user-defined queries will go through ES since it's much simpler to metaprogram queries across denormalized data (no conditional joins).
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#78Are people still using Sphinx Search (http://sphinxsearch.com) at all? It doesn't seem like it gets many releases anymore...since they unpublished the source code, it's hard to see how much activity there is.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#79Elasticsearch, definitely. I always recommend using it in hosted form and not running your own cluster. That allows you to focus on getting data in and out of your cluster instead of sinking time into doing devops. While I have not used Solr recently, it has evolved along with ES as a solid product with a solid community. Nothing against it; it's a solid choice and there are probably people offering to host that as w…
> Elasticsearch, definitely. I always recommend using it in hosted form and not running your own cluster. That allows you to focus on getting data in and out of your cluster instead of sinking time into doing devops. I think everyone doing Elasticsearch well has to bring it in-house eventually. AWS's hosted solution is poor, Logz.io and ElasticCloud are expensive. There's a 7-figure/yr Elastic Cloud customer I work w…
> AWS's hosted solution is poor
What has been poor about it in your experience?
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#80Afaik almost everything runs Lucene under the hood, it's 20 years old, no one is going to build something as good any time soon. I suppose some company like Google have their own in house solution but otherwise it'll always be something built on top of Lucene. I guess you don't see much demand because for a lot of use cases the basic setups are good enough.
About seven years ago, I got a contracting gig for a website that wanted a "search engine". I remember thinking "Solr/Lucene is old, not pure-functional, and therefore awful!" and decided to build my own. Somehow I even managed to convince the client that this was a good idea. I ended up trying to reinvent Solr for the client, realizing after about two days of trying to reinvent stemming and indexing, that this was s…
> "if it works, break it and make it better!"
Fix it 'til it breaks is what I always say :D
Needless to say my 3D printer had a lot of down-time haha.