Understanding Disk Usage in Linux
ownyourbits.com
Understanding Disk Usage in Linux
1–10 of 24 posts
Re: Understanding Disk Usage in Linux
#2Website seems to be hugged to death, here's an archived copy: https://web.archive.org/web/20190429214402/https://ownyourbi...
The article is well worth reading as it's reasonably comprehensive, including a small foray into COW filesystem.
Re: Understanding Disk Usage in Linux
#3To show extents - including sharing - of a file you can use the tool filefrag -v, it's more general than btrfs filesystem du since it uses the fiemap and fibmap ioctls that are supported by multiple, but not all, filesystems.
Re: Understanding Disk Usage in Linux
#4Related: is it possible to reliably maintain physical disk space quotas in Linux (similar to cgroups)?
Furthermore, is it possible to say how much "space" you would use if you were to create a file with a given size, accounting for block-size, fragmentation, and metadata? Matters such as block-size, inode usage, and metadata seem to make this very difficult even if you add special integration to the userspace application, for example by using stat or statfs. This could help prevent quota overruns for example.
These seem like hard problems unfortunately, and I suspect the best solution is to just create separate disk partitions for each quota group.
Re: Understanding Disk Usage in Linux
#5Related: is it possible to reliably maintain physical disk space quotas in Linux (similar to cgroups)? Furthermore, is it possible to say how much "space" you would use if you were to create a file with a given size, accounting for block-size, fragmentation, and metadata? Matters such as block-size, inode usage, and metadata seem to make this very difficult even if you add special integration to the userspace applica…
First question: Quotas have been supported on Linux for a very long time. All major (and native) file systems support them.
Secondly: disk usage accounting for metadata as well as regular file data may or may not be tricky. ZFS always tells you how much data+metadata is used by a file, helped by metadata itself being dynamically allocated on ZFS like everything else is. File systems like ext4 that have fixed metadata locations on disk don't report back metadata allocation with the file; it wouldn't really be useful to see this information since removing the file doesn't free any metadata in the ext4 case.
Re: Understanding Disk Usage in Linux
#6Related: is it possible to reliably maintain physical disk space quotas in Linux (similar to cgroups)? Furthermore, is it possible to say how much "space" you would use if you were to create a file with a given size, accounting for block-size, fragmentation, and metadata? Matters such as block-size, inode usage, and metadata seem to make this very difficult even if you add special integration to the userspace applica…
First question: Quotas have been supported on Linux for a very long time. All major (and native) file systems support them. Secondly: disk usage accounting for metadata as well as regular file data may or may not be tricky. ZFS always tells you how much data+metadata is used by a file, helped by metadata itself being dynamically allocated on ZFS like everything else is. File systems like ext4 that have fixed metadata…
Project quotas appear to be more similar to cgroups. They are available in xfs and ext4 https://lwn.net/Articles/623835/
Re: Understanding Disk Usage in Linux
#7Website seems to be hugged to death, here's an archived copy: https://web.archive.org/web/20190429214402/https://ownyourbi... The article is well worth reading as it's reasonably comprehensive, including a small foray into COW filesystem.
Is that a reference to Hug Bot? https://pbfcomics.com/comics/hug-bot/
Re: Understanding Disk Usage in Linux
#8Website seems to be hugged to death, here's an archived copy: https://web.archive.org/web/20190429214402/https://ownyourbi... The article is well worth reading as it's reasonably comprehensive, including a small foray into COW filesystem.
Is that a reference to Hug Bot? https://pbfcomics.com/comics/hug-bot/
Not sure, but I always thought "hug of death" came from reddit:
Re: Understanding Disk Usage in Linux
#9Questions about "cp --reflink" (I have never used that option so far, but it sounds useful).
Quoting "man":
"When --reflink[=always] is specified, perform a lightweight copy, where the data blocks are copied only when modified. If this is not possible the copy fails"
Q1: this is copy-on-write, right?
Q2: once/if the command completes successfully, are there any (potential) dangers (e.g. if I then immediately delete the original file) or can the new file be treated 100% as if I had copied it the classical way (without that option)?
Thx
Re: Understanding Disk Usage in Linux
#10Questions about "cp --reflink" (I have never used that option so far, but it sounds useful). Quoting "man": "When --reflink[=always] is specified, perform a lightweight copy, where the data blocks are copied only when modified. If this is not possible the copy fails" Q1: this is copy-on-write, right? Q2: once/if the command completes successfully, are there any (potential) dangers (e.g. if I then immediately delete t…
1) Yes, it's copy on write, and requires such support from the underlying filesystem (Btrfs, maybe XFS not sure of what others support it)
2) It's intended to be treated 100% as if it had been copied the classical way. It's not a hardlink or a symlink and can be treated as a completely new file