21–30 of 50 posts
Earlier quoted context omitted.
I have been put off by Elasticsearch's complexity a number of times. Can I ask why with searching a limited about of text you didn't juse use Postgres' full-text search?
I was new to Python, Django and Postgres. I was looking up how to do search and stumbled on articles how to use Elasticsearch in Django. So I went with it.
Earlier quoted context omitted.
I have been put off by Elasticsearch's complexity a number of times. Can I ask why with searching a limited about of text you didn't juse use Postgres' full-text search?
I'm not the person you're replying to, but does Postgres nowadays have a straightforward way to do tf-idf or BM25-style information retrieval?
I commented below - I highly recommend Xapian for small projects to test the waters. It’s the SQLite if search.
3 years ago, I made a simple calendar app in django, and I wanted to use Elasticsearch so users can search and find an event, and to use it to populate an upcoming events list. There's only about 10,000 events in the database. I quickly realized what a pain it is to use Elasticsearch, for a simple app like mine. Pain points: 1) You have to setup and recreate part of your database in elastic search. So you essentially…
Kafka streams can solve this use case fairly well - though setting up & managing infra may be a bit more than what you'd want to deal with for a hobby project
This part left me scratching my head: > We have set “replica 0” in our indexes settings > Now let’s assume that node 3 goes down: > As expected, all shards from node 3 are moved to node 1 and node 2 No, as there are no shards that can be moved, as number of replicas was set to zero and one node went down. Not sure what they are trying to explain here. > In order to resolve this issue, we introduced a job which runs e…
You can set an index template to be used on new indices that match a pattern, which is a very common thing to do. It sounds like what they did was modify the template daily, which is less common IME. It's not clear why they had to manually create the index, though. That should happen automatically.
Earlier quoted context omitted.
I'm not the person you're replying to, but does Postgres nowadays have a straightforward way to do tf-idf or BM25-style information retrieval?
Not op but not that I know of. I commented below - I highly recommend Xapian for small projects to test the waters. It’s the SQLite if search.
Earlier quoted context omitted.
200k documents per second is a lot less impressive, no?
and less catchy.
Earlier quoted context omitted.
Kafka streams can solve this use case fairly well - though setting up & managing infra may be a bit more than what you'd want to deal with for a hobby project
+1, or use any other log-based replication mechanism (e.g. Logstash). The point is that instead of having two independent systems that can easily go out of sync (if not using distributed transactions) and become permanently incostistent with each other, you'll now have the database as the primary source of data (commonly referred to as the system of record) and Elasticseach as a secondary, eventually consistent searc…
Not wanting to do that for a small project, but wanting a better architecture than I've got, I'm curious about your proposed approach.
3 years ago, I made a simple calendar app in django, and I wanted to use Elasticsearch so users can search and find an event, and to use it to populate an upcoming events list. There's only about 10,000 events in the database. I quickly realized what a pain it is to use Elasticsearch, for a simple app like mine. Pain points: 1) You have to setup and recreate part of your database in elastic search. So you essentially…