Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SSDs have become fast, except in the cloud

databasearchitects.blogspot.com

Post reply on HN
Print view View original

281–290 of 427 posts

Re: SSDs have become fast, except in the cloud

#281

Post by bcaxis »

jauntywundrkind wrote: post #254
c0l0 wrote: post #48

Seeing the really just puny "provisioned IOPS" numbers on hugely expensive cloud instances made me chuckle (first in disbelief, then in horror) when I joined a "cloud-first" enterprise shop in 2020 (having come from a company that hosted their own hardware at a colo). It's no wonder that many people nowadays, esp. those who are so young that they've never experienced anything but cloud instances, seem to have little…

NVMe has been ridiculously great. I'm excited to see what happens to prices as E1 form factor ramps up! Much physically bigger drives allows for consolidation of parts, a higher ratio of flash chips to everything else, which seems promising. It's more a value line, but Intel's P5315 is 15TB at a quite low $0.9/GB. It might not help much with oops though. Amazing that we have PCIe 5.0 16GB/s and already are so near th…

The main problem with consumer drives is the missing power loss protection (plp). M.2 drives just don't have space for the caps like an enterprise 2.5 u.2/u.3 drive will have.

This matters when the DB calls a sync and it's expecting the data to be written safely to disk before it returns.

A consumer drive basically stops everything until it can report success and your IOPS falls to like 1/100th of what the drive is capable of if it's happening alot.

An enterprise drive with plp will just report success knowing it has the power to finish the pending writes. Full speed ahead.

You can "lie" to the process at the VPS level by enabling unsafe write back cache. You can do it at the OS level by launching the DB with "eatmydata". You will get the full performance of your SSD.

In the event of power loss you may well end up in an unrecoverable corrupted condition with these enabled.

I believe that if you buy all consumer parts - an enterprise drive is the best place to up spend your money profitably on an enterprise bit.

Re: SSDs have become fast, except in the cloud

#282

Post by EB66 »

Aurornis wrote: post #249
c0l0 wrote: post #48

Seeing the really just puny "provisioned IOPS" numbers on hugely expensive cloud instances made me chuckle (first in disbelief, then in horror) when I joined a "cloud-first" enterprise shop in 2020 (having come from a company that hosted their own hardware at a colo). It's no wonder that many people nowadays, esp. those who are so young that they've never experienced anything but cloud instances, seem to have little…

> It's no wonder that many people nowadays, esp. those who are so young that they've never experienced anything but cloud instances, seem to have little idea of how much performance you can actually pack in just one or two RUs today. On the contrary, young people often show up having learned on their super fast Apple SSD or a top of the line gaming machine with NVMe SSD. Many know what hardware can do. There’s no nee…

> because beyond simple single-purpose applications with flexible uptime requirements, that’s not a realistic option.

I frequently hear this point expressed in cloud vs colo debates. The notion that you can't achieve high availability with simple colo deploys is just nonsense.

Two colo deploys in two geographically distinct datacenters, two active physical servers with identical builds (RAIDed drives, dual NICs, A+B power) in both datacenters, a third server racked up just sitting as a cold spare, pick your favorite container orchestration scheme, rig up your database replication, script the database failover activation process, add HAProxy (or use whatever built-in scheme your orchestration system offers), sprinkle in a cloud service for DNS load balancing/failover (Cloudflare or AWS Route 53), automate and store backups off-site and you're done.

Yes it's a lot of work, but so is configuring a similar level of redundancy and high availability in AWS. I've done it both ways and I prefer the bare metal colo approach. With colo you get vastly more bang for your buck and when things go wrong, you have a greater ability to get hands on, understand exactly what's going on and fix it immediately.

Re: SSDs have become fast, except in the cloud

#283

Post by apitman »

tptacek wrote: post #277
kiwijamo wrote: post #275

Earlier quoted context omitted.

FYI it's not a AWS term, it's a DigitalOcean term.

I could not be more confused. Does EC2 quietly call their hosting machines "droplets"? I knew "droplets" to be a DigitalOcean team, but DigitalOcean doesn't have Nitro cards.

Now I'm wondering if that's where DO got the name in the first place

Re: SSDs have become fast, except in the cloud

#284

Post by Twirrim »

e12e wrote: post #271
Twirrim wrote: post #57

Disclaimer: I work for OCI, opinion my own etc. We offer faster NVMe drives in instances. Our E4 Dense shapes ship with SAMSUNG MZWLJ7T6HALA-00AU3, which supports Sequential Reads of 7000 MB/s, and Sequential Write 3800 MB/s. From a general perspective, I would say the _likely_ answer to why AWS doesn't have faster NVMes at the moment is likely to be lack of specific demand. That's a guess, but that's generally how t…

> I work for OCI Ah: Oracle cloud infra https://blogs.oracle.com/cloud-infrastructure/post/announcin... I keep forgetting Oracle is in the cloud business too. "Make it rain", I guess :)

Doh, sorry. I'm usually good about not using that acronym!

Re: SSDs have become fast, except in the cloud

#285

Post by nimbius »

pclmulqdq wrote: post #11

This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions. SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, whi…

Once again getting a sensible chuckle on hn listening to the cloud crowd whine and complain about what we used to call "the SAN being slow" with the architects argument "the ssds will make things faster"

Re: SSDs have become fast, except in the cloud

#286

Post by kristopolous »

nine_k wrote: post #262
kristopolous wrote: post #243

Earlier quoted context omitted.

You're not considered serious if you don't. Kinda stupid.

In the startup world, this is correct. The success that VCs are after is when your customer base doubles every month. Better yet, every week. Having a reasonably scalable infra at the start ensures that a success won't kill you. Of course, the chances of a runaway success like this are slim, so 99% or more startups overbuild, given their resulting customer base. But it's like 99% or more pilots who put on a parachute…

You aren't going to get there. The risks and complexity of a startup are high to begin with. Adding artificial roadblocks because of aspirational fantasies is going to hold you back.

Look at the big successes such as youtube, twitter, facebook, airbnb, lyft, google, yahoo - exactly zero of them did this preventatively. Even altavista and babelfish, done by DEC and running on Alphas, which they had plenty of, had to be redone multiple times due to growth. Heck, look at the first 5 years of Amazon. AWS was initially ideated in a contract job for Target.

Address the immediate and real needs and business cases, not pie in the sky aspirations of global dominance - wait until it becomes a need and then do it.

The chances of getting there are only reasonable if you move instead of plan, otherwise you'll miss the window and product opportunity.

I know it ruffles your engineering feathers - that's one of the reasons most attempts at building these things fails. The best ways feel wrong, are counterintuitive and are incidentally often executed by young college kids who don't know any better. It's why successful tech founders tend to be inexperienced; it can actually be advantageous if they make the right "mistakes".

Forget about any supposedly inevitable disaster until it's actually affecting your numbers. I know it's hard but the most controllable difference between success and failure in the startup space is in the behavioral patterns of the stakeholders.

Re: SSDs have become fast, except in the cloud

#287

Post by Hewitt821 »

jsnell wrote: post #71
pclmulqdq wrote: post #42

Earlier quoted context omitted.

It's because the "local" SSDs are not actually physically attached and there's a network protocol in the way.

I think you're wrong about that. AWS calls this class of storage "instance storage" [0], and defines it as: > Many Amazon EC2 instances can also include storage from devices that are located inside the host computer, referred to as instance storage. There might be some wiggle room in "physically attached", but there's none in "storage devices located inside the host computer". It's not some kind of AWS-only thing eit…

Local SSD is part of the machine, not network attached.

Re: SSDs have become fast, except in the cloud

#288

Post by tumult »

bcaxis wrote: post #281
jauntywundrkind wrote: post #254

Earlier quoted context omitted.

NVMe has been ridiculously great. I'm excited to see what happens to prices as E1 form factor ramps up! Much physically bigger drives allows for consolidation of parts, a higher ratio of flash chips to everything else, which seems promising. It's more a value line, but Intel's P5315 is 15TB at a quite low $0.9/GB. It might not help much with oops though. Amazing that we have PCIe 5.0 16GB/s and already are so near th…

The main problem with consumer drives is the missing power loss protection (plp). M.2 drives just don't have space for the caps like an enterprise 2.5 u.2/u.3 drive will have. This matters when the DB calls a sync and it's expecting the data to be written safely to disk before it returns. A consumer drive basically stops everything until it can report success and your IOPS falls to like 1/100th of what the drive is c…

My experience lately is that consumer drives will also lie and use a cache, but then drop your data on the floor if the power is lost or there’s a kernel panic / BSOD. (Samsung and others.)

Re: SSDs have become fast, except in the cloud

#289

Post by Hewitt821 »

dekhn wrote: post #126
pclmulqdq wrote: post #42

Earlier quoted context omitted.

It's because the "local" SSDs are not actually physically attached and there's a network protocol in the way.

I suspect you must be conflating several different storage products. Are you saying https://cloud.google.com/compute/docs/disks/local-ssd devices talk to the host through a network (say, ethernet with some layer on top)? Because the documentation very clearly says otherwise, "This is because Local SSD disks are physically attached to the server that hosts your VM. For this same reason, Local SSD disks can only provid…

Local SSD is part of the machine.
Post reply on HN