Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux block devices: hints for debugging and new developments

redhat.com

Post reply on HN
Print view View original

1–5 of 5 posts

Re: Linux block devices: hints for debugging and new developments

#2

Post by rwmj »

Disappointed this doesn't talk about nbdkit, the easiest way to spin up block devices for testing:

https://libguestfs.org/nbdkit-loop.1.html

eg. Make a 7 exabyte disk:

  $ sudo modprobe nbd
  $ nbdkit -U /tmp/sock memory 7E
  $ sudo nbd-client -unix /tmp/sock /dev/nbd0
  $ sudo mkfs.btrfs /dev/nbd0
  $ sudo mount /dev/nbd0 /tmp/mnt
  $ df -h /tmp/mnt
  Filesystem      Size  Used Avail Use% Mounted on
  /dev/nbd0       7.0E  3.4M  7.0E   1% /tmp/mnt
I even did a talk about it:

https://archive.fosdem.org/2019/schedule/event/nbdkit/

Re: Linux block devices: hints for debugging and new developments

#3

Post by project2501a »

rwmj wrote: post #2

Disappointed this doesn't talk about nbdkit, the easiest way to spin up block devices for testing: https://libguestfs.org/nbdkit-loop.1.html eg. Make a 7 exabyte disk: $ sudo modprobe nbd $ nbdkit -U /tmp/sock memory 7E $ sudo nbd-client -unix /tmp/sock /dev/nbd0 $ sudo mkfs.btrfs /dev/nbd0 $ sudo mount /dev/nbd0 /tmp/mnt $ df -h /tmp/mnt Filesystem Size Used Avail Use% Mounted on /dev/nbd0 7.0E 3.4M 7.0E 1% /tmp/mnt…

hot damn, i wanna see mkfs work on a 7 exabyte disk...

Re: Linux block devices: hints for debugging and new developments

#4

Post by rwmj »

project2501a wrote: post #3
rwmj wrote: post #2

Disappointed this doesn't talk about nbdkit, the easiest way to spin up block devices for testing: https://libguestfs.org/nbdkit-loop.1.html eg. Make a 7 exabyte disk: $ sudo modprobe nbd $ nbdkit -U /tmp/sock memory 7E $ sudo nbd-client -unix /tmp/sock /dev/nbd0 $ sudo mkfs.btrfs /dev/nbd0 $ sudo mount /dev/nbd0 /tmp/mnt $ df -h /tmp/mnt Filesystem Size Used Avail Use% Mounted on /dev/nbd0 7.0E 3.4M 7.0E 1% /tmp/mnt…

hot damn, i wanna see mkfs work on a 7 exabyte disk...

mkfs.btrfs is fine - I didn't make that up, that's exactly the output when I ran those commands on my laptop just now. Other filesystems don't handle this situation very well or at all.

Anyway the real power of nbdkit is not that you can create huge disks, but the plugin/filters system, so you can write highly customized block devices for specific use cases, and do things like error injection and test compression or deduplication strategies.

Re: Linux block devices: hints for debugging and new developments

#5

Post by globalc »

rwmj wrote: post #2

Disappointed this doesn't talk about nbdkit, the easiest way to spin up block devices for testing: https://libguestfs.org/nbdkit-loop.1.html eg. Make a 7 exabyte disk: $ sudo modprobe nbd $ nbdkit -U /tmp/sock memory 7E $ sudo nbd-client -unix /tmp/sock /dev/nbd0 $ sudo mkfs.btrfs /dev/nbd0 $ sudo mount /dev/nbd0 /tmp/mnt $ df -h /tmp/mnt Filesystem Size Used Avail Use% Mounted on /dev/nbd0 7.0E 3.4M 7.0E 1% /tmp/mnt…

Ah, valid point, mea culpa. I have not used nbd, but I'm aware of it. Was very impressed to see even a NetBSD implementation. There is just so much around in that area.

In my defense, the storage related PM and colleagues who reviewed the article did not bring up nbd either.

Post reply on HN