31–39 of 39 posts
This sounds just wrong. Why would you use micro service? You are not having teams. Btw. Your data structure/ bytes per row is missing
This sounds just wrong. Why would you use micro service? You are not having teams. Btw. Your data structure/ bytes per row is missing
Having teams is exactly the wrong reason to have micro services, having endpoints with different traffic patterns and resource/scaling constraints is exactly the correct reason.
Earlier quoted context omitted.
Having teams is exactly the wrong reason to have micro services, having endpoints with different traffic patterns and resource/scaling constraints is exactly the correct reason.
Do you really need to jump to micro services when you need scaling? You can also start multiple instances of monoliths/bigger services.
Even otherwise, yes, you sure still can, just like you can write the service in bash if that’s how you want to spend money.
I think you might be asking the wrong questions. They key questions are Queries per second and the median response size of the query. For example at reddit (15 years ago) we had 10x more vote traffic than comment traffic, but we only needed two databases to handle votes (technically only one the other was just for redundancy). But we needed nine comments databases. Mainly because the median query response was so much…
I'm sure latency matters a lot as well. The users of our apps are pretty tolerant of 5 to 10 minute request times for some of our pages, which means we've been able to get away with just a few servers for several TBs of data stored and served. (100+mb responses are not unusual for us). If we had to rethink and redesign the system to cut down those times, we'd need a lot more databases and a much cleverer storage stra…
Earlier quoted context omitted.
I'm sure latency matters a lot as well. The users of our apps are pretty tolerant of 5 to 10 minute request times for some of our pages, which means we've been able to get away with just a few servers for several TBs of data stored and served. (100+mb responses are not unusual for us). If we had to rethink and redesign the system to cut down those times, we'd need a lot more databases and a much cleverer storage stra…
Your users are tolerant of 5-10 minute page request times? Is that a typo?
Earlier quoted context omitted.
Thank you very much! Did you start with separate databases or wait until you needed to scale workloads separately before breaking things up?
I imagine they didn't optimize prematurely.
TiDB, 200k inserts per second; 200b per row on average. Bursty insert pattern, e.g., can have 200k inserts for an hour, then almost 0 for days. 8k reads per second on average, mostly reads by primary key. 20 hosts; 16 threads x 128GB RAM, 8TB NVME RAID 10. 60TiB of useful storage with replication factor of 3. Keyset pagination is the key. Also using rocksb for inserts batching. Costs around 20k on ovh