31–40 of 141 posts
Something that's missing from this which I'm curious about is how far can't postgres search take you? That is, what tends to be the "killer feature" that makes teams groan and set up Elasticsearch because you just can't do it in Postgres and your business needs it? Having dealt with ES, I'd really like to avoid the operational burden if possible, but I wouldn't want to choose an intermediary solution without being ab…
Earlier quoted context omitted.
Hi, I started an Elasticsearch hosting company, since sold, and have built products on PG's search and SQLite FTS search. There are in my mind two reasons to not use PG's search. 1. Elasticsearch allows you to build sophisticated linguistic and feature scoring pipelines to optimize your search quality. This is not a typical use case in PG. 2. Your primary database is usually your scaling bottleneck even without addin…
Do you see any particular reasons to use or not use ZomboDB [1]? It claims to lets you use ElasticSearch from PG seamlessly e.g. it manages coherency of which results ought to be returned according to the current transaction. (I've never quite ended up needing to use ES but it's always seemed to me I'd be likely to need ZomboDB if I did.) [1] https://github.com/zombodb/zombodb
"The only limit, is yourself…"
Earlier quoted context omitted.
Hi, I started an Elasticsearch hosting company, since sold, and have built products on PG's search and SQLite FTS search. There are in my mind two reasons to not use PG's search. 1. Elasticsearch allows you to build sophisticated linguistic and feature scoring pipelines to optimize your search quality. This is not a typical use case in PG. 2. Your primary database is usually your scaling bottleneck even without addin…
Regarding point 2: Shouldn't you be moving your search queries from your transaction server to a separate analysis or read-replica server? OLTP copies to OLAP and suddenly you've separated these two problems.
Earlier quoted context omitted.
Regarding point 2: Shouldn't you be moving your search queries from your transaction server to a separate analysis or read-replica server? OLTP copies to OLAP and suddenly you've separated these two problems.
…and created a new one if the projection is only eventually consistent. No free lunches here.
Something that's missing from this which I'm curious about is how far can't postgres search take you? That is, what tends to be the "killer feature" that makes teams groan and set up Elasticsearch because you just can't do it in Postgres and your business needs it? Having dealt with ES, I'd really like to avoid the operational burden if possible, but I wouldn't want to choose an intermediary solution without being ab…
Exact phrase matching. This generally requires falling back to ILIKE, which is not performant.
Earlier quoted context omitted.
…and created a new one if the projection is only eventually consistent. No free lunches here.
How is that different from running Postgres and Elasticsearch separately?
It’s a difference between having separate OLAP and OLTP databases, which is what the parent post suggested.
Earlier quoted context omitted.
If I recall correctly, Postgres search doesn't scale well. Not sure where it falls apart but it isn't optimized in the same way something like Solr is.
I have a table with over a billion rows and most full-text searches still respond in around a few milliseconds. I think this will depend on a lot of factors, such as proper indexing, and filtering down the dataset as much as possible before performing the full-text ops. I've spent a considerable amount of time on optimizing these queries, thanks to tools like PgMustard [0]. Granted, I do still have a couple slow quer…
Find me some results in my area that contain these categoryIds and are slotted to start between now and next 10 days.
Since its already quite a filtered set of data, would that mean I should have little issues adding pg text search because with correct indexing and all, it will usually be applied to a small set of data?
Thanks
Earlier quoted context omitted.
Do you see any particular reasons to use or not use ZomboDB [1]? It claims to lets you use ElasticSearch from PG seamlessly e.g. it manages coherency of which results ought to be returned according to the current transaction. (I've never quite ended up needing to use ES but it's always seemed to me I'd be likely to need ZomboDB if I did.) [1] https://github.com/zombodb/zombodb
You can do anything, anything at all, at https://zombo.com/ "The only limit, is yourself…"
Earlier quoted context omitted.
…and created a new one if the projection is only eventually consistent. No free lunches here.
How is that different from running Postgres and Elasticsearch separately?
For example, if a place has developed fairly good knowledge of PG already, they can "just" (!) continue developing their PG knowledge.
Adding ES into the mix though, introduces a whole new thing that needs to be learned, optimised, etc.