Live data from Hacker News

Ask HN: Are Lucene/Solr/ES Still Used for Search?

news.ycombinator.com

81–90 of 223 posts

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#81

Search built using Postgres is underrated. It can do a lot if used properly.

there are on different levels, a search on a sql database will have real time results, while on solr / elasticsearch it's going to have a delay (from milliseconds to minutes). That delay gives the advantage to build a series of data structures much more suited for search than the ones on a database.

I built several search systems for classified listing sites, something like solr is a life saver once you get enough traffic. Is much easier to scalate than a sql database, and you can do much more things. The easiest example is a facet, for example you make a search on a car listing site, and you want to show how many cars from each brand you are matching, with sql you have to make another query, while in solr you can get those results in the same query. Now, add the model, the color, the gas type, the transmission, the place, etc. that actually grows easily to something unescalable, while with solr you can do easily.

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#83

We have been using https://www.algolia.com/ completely as a replacement for ES. Pros: - Managed search engine - Great API / Developer experience Cons: - Cloud only makes it hard for local development - Expensive (I guess it depends on the usage)

It is a similar cost to elastic/solr cloud options, cheaper if you have to get to feature parity with Algolia.

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#84
post #6

Afaik almost everything runs Lucene under the hood, it's 20 years old, no one is going to build something as good any time soon. I suppose some company like Google have their own in house solution but otherwise it'll always be something built on top of Lucene. I guess you don't see much demand because for a lot of use cases the basic setups are good enough.

Wonder what DuckDuckGo might have built for its search..

Word is they use bing api for web search.

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#85
post #78

Are people still using Sphinx Search ( http://sphinxsearch.com ) at all? It doesn't seem like it gets many releases anymore...since they unpublished the source code, it's hard to see how much activity there is.

We just migrated from Sphinx to using the full text search indexes in PostgreSQL, we had to deal with some changes in how special characters are handled, but it's worked well enough.

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#86
post #66

Earlier quoted context omitted.

See my reply to bryanrasmussen for a full explanation. Basically: you reindex ES periodically, so when a user is deleted from the primary, it will disappear from ES upon the next reindex. The old index is deleted at the file system level.

At some point, though, the pedantry can get out of hand. After all, 'deleting' at the file system level is just 'unlinking' the inode from the underlying data blocks... in fact, data forensics at the file system level is probably more well-understood than recovering deleted data from a Lucene shard. at what point would you be able to 'delete' data without being in violation of GDPR?

I know - it really is a matter of definition.

Though the EU has said it will consider intention etc. there's really no way of knowing for certain until when and if it's settled in a court case.

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#87

AFAIK, Reddit, Slack, Dice, Bloomberg, IBM, Apple all use Solr. Jira and Confluence use Lucene. Others use Elasticsearch and Fusion (commercial product on top of Solr). See, for example: https://www.activate-conf.com/more-events for presentations from several past years on who and how uses Lucene/Solr. Also, the new trend in jobs is "Relevancy Engineering", which is less about just setting up search engines and more…

From what I read, wikipedia uses ES.

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#88
post #17
post #6

Afaik almost everything runs Lucene under the hood, it's 20 years old, no one is going to build something as good any time soon. I suppose some company like Google have their own in house solution but otherwise it'll always be something built on top of Lucene. I guess you don't see much demand because for a lot of use cases the basic setups are good enough.

About seven years ago, I got a contracting gig for a website that wanted a "search engine". I remember thinking "Solr/Lucene is old, not pure-functional, and therefore awful!" and decided to build my own. Somehow I even managed to convince the client that this was a good idea. I ended up trying to reinvent Solr for the client, realizing after about two days of trying to reinvent stemming and indexing, that this was s…

I can chime in here that lucene- based solutions are sufficient almost always, for a purely frontend, js-based fuzzy search engine check out fusejs. https://fusejs.io/

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#89

About a year ago I set up an ES cluster to load Apache logs into as an experiment. Around a month later my boss asked if it was down. "Yeah, it looks like it, since you are using it let me upgrade it from an experiment to critical!" Since then we've been using it in more and more places, and are looking to see if it would be a good fit for storing some of our user data in. The big barrier right now is that we are run…

>The SQL server is always going to be our source of truth, probably. ES is not supposed to be your source of truth.

In the early days at least, cosmosdb was built on es. Dunno about now though.

Re: Ask HN: Are Lucene/Solr/ES Still Used for Search?

#90
post #27

Very much so. For retail/catalog search SOLR dominates. There's a lot more customization available for relevancy/ranking OOB than Elastic. Drawbacks are managing indexing - SOLR cloud is much harder to manage. For commodity search workloads (general retrieval/faceting) Elastic does a fine job. It scales well and there is good documentation and support. Lucene is the core engine behind both of these solutions. For fun…

Vivisimo was actually bought by IBM, not Oracle.
Post reply on HN