31–40 of 92 posts
Earlier quoted context omitted.
From a database perspective, 1ms to disk is an eternity. A good disk subsystem had less write latency than that in the early 90’s.
I don’t think that’s true. I recall ~5ms seek times being top of the line.
what was the latency to the controller with ram cache?
using seek time as a measure is also somewhat worst case - controllers/filesystems also queue(d) according to drive geometry.
Kind of feels like the author doesn’t know kubernetes? Stateful sets make all this very easy. I only have a user-end knowledge of k8s from GCE and it seems like k8s makes storage very easy. EDIT K8s makes storage very easy. No “seems.”
or, using storage in an environment where the real storage is completely managed for you makes storage seem easy..
I don't really think this is a Kubernetes-specific problem. If you have a million machines, want your database to run on one of them that is selected by some upstream orchestrator, and want the physical SSDs with that data on it to be in the same machine, you're going to have to do some work. But at the same time, you have to realize that you are doing this to get that tiny last bit of performance (most likely that 9…
For databases, local storage beats SAN storage massively. I can get better performance and IOPS from an Intel NUC with a decent PCIe SSD than I can get from an AWS RDS instance that costs as much per month as the NUC did to buy. If I optimize a proper rack mount server for database I can get an insane amount and storage and performance compared to even a few months of RDS. Even a pair of 40 Gbps SAN links would not c…
Only if your (storage) network is slow (eg 1/10/25 GbE).
With a decent network (eg modern infiniband, 40+GbE, etc) for the storage, the latency and throughput to the storage shouldn't make a difference.
For example (years ago), I used to set up SSD arrays - SATA at the time, as M.2 wasn't a thing - and have them served over a 20Gbs Infiniband network to hosts in the same data centre.
The access times from an OS perspective to hit that storage over the network were the same as for hitting local disk. But the networked storage was higher bandwidth (multiple SSD's, instead of a single per host).
Worked really well. :)
Here's the root of the problem. > Static provisioning also goes against the mindset of Kubernetes Then the mindset of Kubernetes is wrong. Or at least incomplete. Persistent data is an essential part of computing. In some ways it's the most important part. You could swap out every compute element in your system and be running exactly the way you were very quickly. Now try it with your storage elements. Oops, screwed.…
The tail wagging the dog that is the last 5 years of devops advancement. Constraints imposed on software development to achieve reproducibility. The most obvious example of this is the need to prematurely scale horizontally and deal with ephemeraliy (even compute).
Earlier quoted context omitted.
The tail wagging the dog that is the last 5 years of devops advancement. Constraints imposed on software development to achieve reproducibility. The most obvious example of this is the need to prematurely scale horizontally and deal with ephemeraliy (even compute).
If you’re ever planing on running at scale (and I assume most startups/projects strive for it), then there’s no point (other than maybe a rough poc) at which sharding is “premature” because later on it most likely will be prohibitively expensive.
Also, whatever unnecessary scaling solution you come up with today for the problems you have today is not likely to be applicable to the problems you have years down the road; features will have changed, apps will be rewritten.
I think people underestimate how much a handful of properly specd servers can achieve. My best experience with "at-scale" was 25+ million users, heavy API usage (and the API hits weren't trivial / hard to cache), and we could handle all the load on 2 beefy servers (but had more for HA and lower latency (they were distributed)).
Quick search and I found a dual EPYC 7281 with 256GB DDR4, 2TB nvme, 2TB ssd, and 12TB HDD for $500/m in LA. Now it obviously depends on what you're doing, but for most startups to max this out, they're either extremely successful or extremely bad software engineers.
Earlier quoted context omitted.
For databases, local storage beats SAN storage massively. I can get better performance and IOPS from an Intel NUC with a decent PCIe SSD than I can get from an AWS RDS instance that costs as much per month as the NUC did to buy. If I optimize a proper rack mount server for database I can get an insane amount and storage and performance compared to even a few months of RDS. Even a pair of 40 Gbps SAN links would not c…
I'm not sure this is necessarily true. 2 x 40Gbps is bandwidth which is typically not the limiting factor. If it was you can go with higher bandwidth like FC. RDS is a database service, not SAN. Look at SAN boxes from storage companies, you can get things like 60 SSDs in a single box. To match the IOPs of that you'd need a lot of servers with local storage. I think in general dis-aggregating compute and storage is th…
Service Fabric powers many Microsoft services today, including Azure SQL Database, Azure Cosmos DB, Cortana, Microsoft Power BI, Microsoft Intune, Azure Event Hubs, Azure IoT Hub, Dynamics 365, Skype for Business, and many core Azure services.
https://blogs.msdn.microsoft.com/azuredev/2018/08/15/service...
Earlier quoted context omitted.
For databases, local storage beats SAN storage massively. I can get better performance and IOPS from an Intel NUC with a decent PCIe SSD than I can get from an AWS RDS instance that costs as much per month as the NUC did to buy. If I optimize a proper rack mount server for database I can get an insane amount and storage and performance compared to even a few months of RDS. Even a pair of 40 Gbps SAN links would not c…
I'm not sure this is necessarily true. 2 x 40Gbps is bandwidth which is typically not the limiting factor. If it was you can go with higher bandwidth like FC. RDS is a database service, not SAN. Look at SAN boxes from storage companies, you can get things like 60 SSDs in a single box. To match the IOPs of that you'd need a lot of servers with local storage. I think in general dis-aggregating compute and storage is th…
Earlier quoted context omitted.
If you’re ever planing on running at scale (and I assume most startups/projects strive for it), then there’s no point (other than maybe a rough poc) at which sharding is “premature” because later on it most likely will be prohibitively expensive.
Picking sharding specifically is weird given the many problem spaces where it won't work. Also, whatever unnecessary scaling solution you come up with today for the problems you have today is not likely to be applicable to the problems you have years down the road; features will have changed, apps will be rewritten. I think people underestimate how much a handful of properly specd servers can achieve. My best experie…
And if you're willing to actually get a physical server, Dell happily sells single nodes with 6 terabytes of RAM for a few tens of thousands. Other vendors go much higher.
Earlier quoted context omitted.
The tail wagging the dog that is the last 5 years of devops advancement. Constraints imposed on software development to achieve reproducibility. The most obvious example of this is the need to prematurely scale horizontally and deal with ephemeraliy (even compute).
If you’re ever planing on running at scale (and I assume most startups/projects strive for it), then there’s no point (other than maybe a rough poc) at which sharding is “premature” because later on it most likely will be prohibitively expensive.
In the past, I've supported a few million daily active users on a single MySQL database.
Use cases obviously vary, your numbers may not line up.
But I'm pretty sure there are lot of startups that would be fine starting with an architecture that can scale to 2 million users per day.