71–77 of 77 posts
Earlier quoted context omitted.
just wondering why Mongo is considered a part of "JS". Is it because of the MEAN stack?
Part of it is JSON as the storage format. Another part is its Node driver. The whole API fit it well. It understood async programming. It felt JS-like. The input and output were JSON instances. Finally, yea, from what I know of him "web scale" did play part in the decision. Oh those heady days.
it has first class support in most ORM, and works quite well.
Earlier quoted context omitted.
FWIW we went through a very similar process to that documented here by Github (~3 months ago). It was entirely due to operational reasons and nothing to with shortcomings in Redis itself. MySQL was the master record for 99% of our data while Redis was the master record for the other 1% (as it happens it was also a kind of activity stream). Having the single 'master' reference for our data reduced complexity to a degr…
just asking for some info, but how do you make sure that multiple of your db systems are in sync (specifically interested in MySql and elasticsearch)? Hope it's alright to ask you that.
Pgsql is a bit harder, but if I needed to start somewhere it would be with:
https://github.com/debezium/debezium
or https://github.com/confluentinc/bottledwater-pg
These are the start of pretty sophisticated solutions where you need super real-time elasticsearch indexes and can bring up infra like Kafka.
For many applications, queueing an update when something hits your ORM to update, with the hourly/daily refresh is pretty satisfactory.
Has Github stopped allowing free searching for code? All I get is "Must include at least one user, organization, or repository". I think that its a greater problem than the speed of its streams.
Wow I'm learning today that Github used Redis for persistent data, now that they moved away :-) Anyway very happy that Redis helped to run such an important site. From the blog post it looks like that for certain things to move away from Redis was hard even if they are very skilled with MySQL, this is a good thing from the POV of Redis since it means that Redis allows to model certain things easily. However they want…
FWIW we went through a very similar process to that documented here by Github (~3 months ago). It was entirely due to operational reasons and nothing to with shortcomings in Redis itself. MySQL was the master record for 99% of our data while Redis was the master record for the other 1% (as it happens it was also a kind of activity stream). Having the single 'master' reference for our data reduced complexity to a degr…
We have nearly everything in Postgres, and redis serves as both caching layer (non-persistent), but also for rails session storage and Sidekiq (persistent).
Having one source of truth can make things like failover much easier. I can handle PG failover, and also redis, but I'd rather not have to deal with both. Especially if you consider the potential of things going slightly out-of-sync (think a job in sidekiq that relies on an id in PG, one of which loses a few microseconds of data during replication etc, just speculating a scenario here)
Did anybody face similar challenges and care to share their thoughts?
Can anybody here help me understand why many teams are using MySQL as a KV store? (Uber did it recently, so assuming many others probably did it too, network effect) I personally love MySQL. Just want to understand what makes MySQL a great KV store as opposed to more seemingly specialized systems like Redis?
> Just want to understand what makes MySQL a great KV store Its not, it just happens to be good enough, which matters a lot for operational expertise/costs/etc. For example, you can store hundreds of millions of KV rows in an InnoDB table and still have <1-3ms response times on queries, while having persistence built in. Perfect is the enemy of good enough.
I don't know why people use redis as an LRU cache. Its a terrible LRU cache. Its eviction algorithm isn't true LRU and does sampling which may cause new keys to get incorrectly convicted. LRU is also really slow being single threaded.
What should people use?