ES is used in code search tool dxr which we locally use as well: http://dxr.mozilla.org
Ask HN: Are Lucene/Solr/ES Still Used for Search?
101–110 of 223 posts
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#102Very much so. For retail/catalog search SOLR dominates. There's a lot more customization available for relevancy/ranking OOB than Elastic. Drawbacks are managing indexing - SOLR cloud is much harder to manage. For commodity search workloads (general retrieval/faceting) Elastic does a fine job. It scales well and there is good documentation and support. Lucene is the core engine behind both of these solutions. For fun…
There is a long term trend of search engine companies being acquired, going away, products ending, and customers left in the lurch....
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#103Are people still using Sphinx Search ( http://sphinxsearch.com ) at all? It doesn't seem like it gets many releases anymore...since they unpublished the source code, it's hard to see how much activity there is.
We just migrated from Sphinx to using the full text search indexes in PostgreSQL, we had to deal with some changes in how special characters are handled, but it's worked well enough.
Do you have any numbers on your requests or searches per second in a real use case? I've really been wondering this as I've been considering manticore which is the major sphinx fork.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#104Earlier quoted context omitted.
Definitely. Product catalog information as a whole doesn't change often. Price and availability does. With retail catalogs you often do a full reindex of the data in your master catalog and then run partials to account for price/availability if you don't do that in realtime using filters. Since the system of record is not the search index, Elastic is often not a good solution here. Also, relevancy in retail is often…
One more point - if you can run the entire index on one machine it makes deploying and managing SOLR much easier to manage than Elastic. The complexity only grows when you have a distributed system. You can fit a lot on a big box.
Yes, it's certainly easy to manage on one system.
As the production system I work on is an academic one with few concurrent users it's possible to get away with that.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#105Afaik almost everything runs Lucene under the hood, it's 20 years old, no one is going to build something as good any time soon. I suppose some company like Google have their own in house solution but otherwise it'll always be something built on top of Lucene. I guess you don't see much demand because for a lot of use cases the basic setups are good enough.
>Afaik almost everything runs Lucene under the hood, it's 20 years old, no one is going to build something as good any time soon.
Vespa [1] would like to have a word with you.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#106Earlier quoted context omitted.
About seven years ago, I got a contracting gig for a website that wanted a "search engine". I remember thinking "Solr/Lucene is old, not pure-functional, and therefore awful!" and decided to build my own. Somehow I even managed to convince the client that this was a good idea. I ended up trying to reinvent Solr for the client, realizing after about two days of trying to reinvent stemming and indexing, that this was s…
> "if it works, break it and make it better!" Fix it 'til it breaks is what I always say :D Needless to say my 3D printer had a lot of down-time haha.
"If it ain't broke, open it up and find out what makes it so bloody special" - i think that's wisdom from the BOFH.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#107We use elasticsearch to power our ecommerce search, and it works pretty well, but we're considering moving to a commercial product, or solr, to get closer to personalized results based on our knowledge about the user.
We just rewrote our internal search API from a windows service indexer with lucene indices and a vb.net SOAP api in iis to a netcore service, hosted in k8s, that splits out ingest, analysis, storage and queries into separate domains, with the writes going to Azure Search Service*.
Our use case might be a bit weird -- this app is essentially an internal API that supports the search needs of our other teams and their own products for our own internal software. It probably has 30 million records across a few different indices. We made the decision to migrate from lucene because of the ease of clustering elasticsearch. We previously achieved availability by just running multiple copies of the standalone service and doing smart health checks at the load balancer level in case a lucene index got corrupted and needed to spend a day rebuilding, but that didn't scale well for rebuild times, and we have been consolidating all of our legacy tech onto netcore and kubernetes.
Raw lucene was an order of magnitude faster than azure search service, but that's probably more a function of being able to essentially query the indices directly in memory of the webservice, as opposed to a slightly underprovisioned search cluster with all the HTTP overhead. We're migrating it to our own elasticsearch cluster right now for performance, cost savings and cloud-agnosticism.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#108About a year ago I set up an ES cluster to load Apache logs into as an experiment. Around a month later my boss asked if it was down. "Yeah, it looks like it, since you are using it let me upgrade it from an experiment to critical!" Since then we've been using it in more and more places, and are looking to see if it would be a good fit for storing some of our user data in. The big barrier right now is that we are run…
>The SQL server is always going to be our source of truth, probably. ES is not supposed to be your source of truth.
Can you elaborate on the reasons for this?
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#109Earlier quoted context omitted.
What do you mean by saying scaling Elastic is a dark art? Care to expand on that?
I'm by no means heavily experienced in this, but my current employer has a big demand for this. (e-commerce) First there's the Docker + Kubernetes architecture that ES lends itself to really well. Then (depending on your use-case) there are concerns like hot/warm architecture, node types, ETL/indexing processes. ES recently moved over to openJDK, so there's a couple intricacies there (i.e. JVM heap size) Then, there'…
> ES recently moved over to openJDK, so there's a couple intricacies there (i.e. JVM heap size)
My current employers uses ES - we're on 6.8, planning to move to 7 in a few months. Judging by the other replies here I'd say we have a reasonably large cluster (150+ i3.2xlarge instances, billions of documents), so tuning the cluster is very relevant to us. Could you expand on how things have changed with the move to OpenJDK?
I've seen some claims online that, contrary to what Elastic recommends in their docs, a few machines with huge heaps (100+ gb) is the way to go, rather than many machines with 20gb heaps.
Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?
#110We have been using https://www.algolia.com/ completely as a replacement for ES. Pros: - Managed search engine - Great API / Developer experience Cons: - Cloud only makes it hard for local development - Expensive (I guess it depends on the usage)
Worth noting this is what hacker news itself uses
you mean https://hn.algolia.com/? thats not made by the HN team though, wouldn't be surprised if the algolia guys built it to show their tech, which is a very smart idea anyway.