IO Devices and Latency • Hacker News Board

Re: IO Devices and Latency

#21

Post by lizztheblizz » Thu, Mar 13, 2025, 6:17 PM UTC

alexellisuk wrote: post #13

bddicken wrote: post #3

Author of the blog here. I had a great time writing this. By far the most complex article I've ever put together, with literally thousands of lines of js to build out these interactive visuals. I hope everyone enjoys.

Hi, what actually are _metal_ instances that are being used when you're on EC2 that have local NVME attached? Last time I looked, apart from the smallest/slowest Graviton, you have to spend circa 2.3k USD/mo to get a bare-metal instance from AWS - https://blog.alexellis.io/how-to-run-firecracker-without-kvm...

Hi there, PS employee here. In AWS, the instance types backing our Metal class are currently in the following families: r6id, i4i, i3en and i7ie. We're deploying across multiple clouds, and our "Metal" product designation has no direct link to Amazon's bare-metal offerings.

Re: IO Devices and Latency

#22

Post by bloopernova » Thu, Mar 13, 2025, 6:20 PM UTC

Fantastic article, well explained and beautiful diagrams. Thank you bddicken for writing this!

Re: IO Devices and Latency

#23

Post by bddicken » Thu, Mar 13, 2025, 6:21 PM UTC

bloopernova wrote: post #22

Fantastic article, well explained and beautiful diagrams. Thank you bddicken for writing this!

You are welcome!

Re: IO Devices and Latency

#24

Post by TechDebtDevin » Thu, Mar 13, 2025, 6:22 PM UTC

bddicken wrote: post #23

bloopernova wrote: post #22

Fantastic article, well explained and beautiful diagrams. Thank you bddicken for writing this!

You are welcome!

Probably the best diagrams I've ever seen in a blog post.

Re: IO Devices and Latency

#25

Post by magicmicah85 » Thu, Mar 13, 2025, 6:23 PM UTC

Can I just say that I love how informative this was that I completely forgot it was to promote a product? Excellent visuals and interactivity.

Re: IO Devices and Latency

#26

Post by jstimpfle » Thu, Mar 13, 2025, 6:25 PM UTC

bob1029 wrote: post #18

I've been advocating for SQLite+NVMe for a while now. For me it is a new kind of pattern you can apply to get much further into trouble than usual. In some cases, you might actually make it out to the other side without needing to scale horizontally. Latency is king in all performance matters. Especially in those where items must be processed serially. Running SQLite on NVMe provides a latency advantage that no other…

I still measure 1-2ms of latency with an NVMe disk on my Desktop computer, doing fsync() on a file on a ext4 filesystem.

Update: about 800us on a more modern system.

Re: IO Devices and Latency

#27

Post by gozzoo » Thu, Mar 13, 2025, 6:25 PM UTC

Can someeone share their expirience in creating such diagrams. What libraries and tools can be useful for such interactive diagrams?

Re: IO Devices and Latency

#28

Post by Joel_Mckay » Thu, Mar 13, 2025, 6:29 PM UTC

gozzoo wrote: post #27

Can someeone share their expirience in creating such diagrams. What libraries and tools can be useful for such interactive diagrams?

Do you mean something for data visualization, or tricks condensing large data sets with cursors?

https://d3js.org/

Best of luck =3

Re: IO Devices and Latency

#29

Post by aftbit » Thu, Mar 13, 2025, 6:29 PM UTC

Hrm "unlimited IOPS"? I suppose contrasted against the abysmal IOPS available to Cloud block devs. A good modern NVMe enterprise drive is specced for (order of magnitude) 10^6 to 10^7 IOPS. If you can saturate that from database code, then you've got some interesting problems, but it's definitely not unlimited.

Re: IO Devices and Latency

#30

Post by bddicken » Thu, Mar 13, 2025, 6:30 PM UTC

gozzoo wrote: post #27

Can someeone share their expirience in creating such diagrams. What libraries and tools can be useful for such interactive diagrams?

For this particular one I used d3.js, but honestly this isn't really the type of thing it's designed for. I've also used GSAP for this type of thing on this article I wrote about database sharding.

https://planetscale.com/blog/database-sharding