Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SSDs have become fast, except in the cloud

databasearchitects.blogspot.com

Post reply on HN
Print view View original

401–410 of 427 posts

Re: SSDs have become fast, except in the cloud

#401

Post by eek2121 »

fcsp wrote: post #344
jsnell wrote: post #199

Earlier quoted context omitted.

I've linked to public documentation that is pretty clearly in conflict with what you said. There's no wiggle room in how AWS describes their service without it being false advertising. There's no "ah, but what if we define the entire building to be the host computer, then the networked SSDs really are inside the host computer" sleight of hand to pull off here. You've provided cryptic hints and a suggestion to watch s…

I see wiggle room in the statement you posted in that the SSD storage that is physically inside the machine hosting the instance might be mounted into the hypervised instance itself via some kind of network protocol still, adding overhead.

At minimum, the entire setup will be virtualized, which does add overhead.

Re: SSDs have become fast, except in the cloud

#402

Post by bcaxis »

silent_cal wrote: post #374
bcaxis wrote: post #371

Earlier quoted context omitted.

This matches my experience. Something about cloud systems makes them incredibly slow compared to real hardware. Not just the disk, the CPU is more limited too. I fire up vCPU or dedicated or bare metal in the cloud, doesn't matter, I simply cannot match the equivalent compute of real hardware and it's not even close.

Isn't that expected? I would assume cloud stuff is slower because it's essentially an emulation of the real thing.

When I spin up a vm on my hardware and run an application. The performance is generally about 70% of what I can get in a container running on an OS on that very same bare metal.

But that isn't the delta I'm seeing, it's 5-10x performance delta not a 30-50% delta.

Re: SSDs have become fast, except in the cloud

#403

Post by vitus »

scottlamb wrote: post #392
vitus wrote: post #263

Earlier quoted context omitted.

> The snap paper linked elsewhere in the thread shows 10 µs latencies; they've put in some optimization to achieve that, but I don't really understand why we don't expect close to that with standard kernel networking and TCP. You can get similar results by looking at comparisons between DPDK and kernel networking. Most of the usual gap comes from not needing to context-switch for kernel interrupt handling, zero-copy…

Thanks for the link. How does this compare to the analogous situation for SSD access? I know there are also userspace IO stacks for similar reasons, but it seems like SSD-via-kernel is way ahead of network-via-kernel in the sense that it adds less latency per operation over the best userspace stack.

I can't say offhand, since I don't have much experience with storage.

I'd expect that most of the work from a SSD read is offloaded to the disk controller, which presumably uses DMA, and you don't have nearly as many round trips (a sequential read can be done with a single SCSI command).

I'm inclined to agree with the explanation given by other commenters that the limiting factor for SSD r/w speeds in the cloud is due to throttling in the hypervisor to provide users with predictable performance as well as isolation in a multitenant environment.

Re: SSDs have become fast, except in the cloud

#404

Post by jiggawatts »

wistlo wrote: post #377
rcarmo wrote: post #372

Earlier quoted context omitted.

As someone who works with Azure daily, I am amazed not just at the initial consultant's conclusion (that is, alas, typical of folk who do not understand database engines), but also to your struggle with NVMe storage (I have some pretty large SQLite databases on my personal projects). You should not have needed an Ebsv5 (memory-optimised) instance. For that kind of thing, you should only have needed a D-series VM with…

I may have the "Ebsv5" series code incorrect. I'd look it up, but I don't have access to the subscription any longer. What I chose ultimately was definitely "nVME attached" and definitely pricey. The "hypervisor-adjacent, very low latency volume" was not an obvious choice. The best performing configuration did come from me--the db admin learning Azure on the fly--and not the four Azure architects nor the half dozen c…

Ebsv5 and Ebdsv5 somewhat uniquely provide the highest possible storage performance right now in Azure, partly because they support NVMe controllers instead of SCSI.

However, the disks are still remote replicas sets as someone else mentioned. They’re not flash drives plugged into the host, despite appearances.

Something to try is (specifically) the Ebdsv5 series with the ‘d’ meaning it has local SSD cache and temp disks. Configure Postgres to use the temp disk for its scratch space and turn on read/write caching for the data disks.

You should see better performance, but still not as good as a laptop… that will have to wait for the v6 generation of VMs.

Re: SSDs have become fast, except in the cloud

#405

Post by motles »

wistlo wrote: post #364

At my job at a telco, I had a 13 billion record file to scan and index for duplicates and bad addresses. Consultants brought in to move our apps (some of which were Excel macros, others SAS scripts running on old desktop) to Azure. The Azure architects identified Postgres as the best tool. Consultants attempted to create a Postgres index in a small Azure instance but their tests would fail without completion (they we…

Was this on the L-series or managed disk?

Re: SSDs have become fast, except in the cloud

#406

Post by motles »

Too wrote: post #395
wistlo wrote: post #364

At my job at a telco, I had a 13 billion record file to scan and index for duplicates and bad addresses. Consultants brought in to move our apps (some of which were Excel macros, others SAS scripts running on old desktop) to Azure. The Azure architects identified Postgres as the best tool. Consultants attempted to create a Postgres index in a small Azure instance but their tests would fail without completion (they we…

Ebsv5 does not have a local nvme. To get local drive you need to pick the version with a “d” in the name.

Lsv3 series

Re: SSDs have become fast, except in the cloud

#407

Post by asalahli »

wmf wrote: post #329
phanimahesh wrote: post #322

Earlier quoted context omitted.

The parent claims that though aws uses better hardware, they bill in vcpus whose benchmarks are from a few years ago, so that they can sell more vcpu units per performant physical cpu. This does not contradict your claim that aws buys better hardware.

It's so obviously wrong that I can't really explain it. Maybe someone else can. To believe that requires a complete misunderstanding of IaaS.

GP is probably refferring to blurbs like these

> Amazon SimpleDB measures the machine utilization of each request and charges based on the amount of machine capacity used to complete the particular request (SELECT, GET, PUT, etc.), normalized to the hourly capacity of a circa 2007 1.7 GHz Xeon processor. See below for a more detailed description of how machine utilization charges are calculated.

https://aws.amazon.com/simpledb/pricing/

Re: SSDs have become fast, except in the cloud

#408

Post by yolovoe »

sargun wrote: post #233
tptacek wrote: post #215

Earlier quoted context omitted.

"Attached to the droplet"?

Droplets are what EC2 calls their hosts. Confusing? I know.

Yes, we internally call servers droplets. We have multiple hosts/mobos in the same server these days so calling them hosts is confusing, and droplet is a really old term here from what i can tell.

Re: SSDs have become fast, except in the cloud

#409

Post by yolovoe »

kiwijamo wrote: post #275
tptacek wrote: post #235

Earlier quoted context omitted.

Yes! That is confusing! Tell them to stop it!

FYI it's not a AWS term, it's a DigitalOcean term.

It’s an internal ec2 term too. We don’t use it externally and I shouldn’t have used it to avoid all this confusion.

Internally, we say droplet instead of host as there are multiple hosts/mobos per droplet these days. It’s no longer true that when you get a metal droplet, you get the entire droplet.

Re: SSDs have become fast, except in the cloud

#410

Post by yolovoe »

tptacek wrote: post #215
yolovoe wrote: post #195

Earlier quoted context omitted.

You’re wrong. Instance local means SSD is physically attached to the droplet and is inside the server chassis, connected via PCIe. Sourece: I work on nitro cards.

"Attached to the droplet"?

I shouldn’t have said droplet. Like sibling says, that’s our internal name for a “server” and not what we use externally.
Post reply on HN