61–70 of 77 posts
Earlier quoted context omitted.
"After a lot of hunting around, I found out that I had hit my inode limit, a concept I was completely unaware of." What was the number you hit? I have a flat file system for something I'm prototyping. The files are in a database with a random SHA256 filename associated with them. I was thinking about doing what Git does with creating folders for parts of the string to alleviate the number of files in one single direc…
It's a global limit per filesystem. You can check what the limits are for your filesysems, and what the current use is with `df -i`. The thing that git does is for speed.
https://developer.apple.com/legacy/library/documentation/Dar...
Maybe Linux and the BSDs could adopt that as well.
Earlier quoted context omitted.
It's a property of the filesystem that is set when it was created. In the case of the mkfs.ext{2,3,4}, you can set the number of inodes with the -N flag, but by default, it is the size of the filesystem divided by inode_ratio (probably 4096).
This seems to be one of those bits of old UNIX lore that's being lost - it used to be fairly well-known in sysadmin circles that if you were going to be storing a lot of tiny files, you should create your filesystem with a higher than usual number of inodes (the classic case was an NNTP spool mount).
I wonder how many people understand what the term "news server" of the partition wizard in the Debian installer means.
ext4 doesn't care if there's collisions in the dir_index hash - they're entirely expected. The dir_index hashes are just used to look up in a tree, the lowest level of which points you to the directory block that contains all the directory entries for names with hashes in a given range.
However, those hashes do get used for the directory position used by telldir() / seekdir() - and those are only 32 bits on 32 bit architectures. So I suspect that something involving those calls is generating the ENOSPC.
Brings back memories of the old DOS days, when there was quite a low limit on the number of files that you could store in the root directory. I can't remember the exact numbers, but it lead to my early habits of creating a new sub directory for every little thing so nowadays (over 3 decades later) nearly everything I need is at least 5 levels deep! ;) NB: My current working folders are now in Google Drive, but is ess…
.. aided by CHKDSK, which would find orphaned allocation units and create files for them in the root directory. Could quite easily fill up your root directory on its own.
Statements like this make me trust a developer significantly less. It's our job to care about "technical reasons."
Things like this is why I use XFS anywhere I'm not using btrfs, or ZFS.
The reason I stay with ext4 is that it has proven amazingly robust in the event of hard reboots.
Earlier quoted context omitted.
It's a property of the filesystem that is set when it was created. In the case of the mkfs.ext{2,3,4}, you can set the number of inodes with the -N flag, but by default, it is the size of the filesystem divided by inode_ratio (probably 4096).
This seems to be one of those bits of old UNIX lore that's being lost - it used to be fairly well-known in sysadmin circles that if you were going to be storing a lot of tiny files, you should create your filesystem with a higher than usual number of inodes (the classic case was an NNTP spool mount).
That lead to the second discussion of why you don't fix it by piping filenames one by one to rm, but either use find's inbuilt unlink/rm if there is one or xargs to rm if not.
Earlier quoted context omitted.
Your filesystem's block size is probably 4KiB. So, it's one inode per for blocks. That's crazy. Why? Do you expect to store many files less than 16KiB?
Each user has some small arbitrary information associated with them, with potentially tens of millions of users (and no doubt more in the future). There's not really a structure due to the nature of the project, each user needs personalised information and structure associated to them. I don't think that amount of information is really suitable for a database? It may be larger in some cases too, so there's not guaran…
Earlier quoted context omitted.
This seems to be one of those bits of old UNIX lore that's being lost - it used to be fairly well-known in sysadmin circles that if you were going to be storing a lot of tiny files, you should create your filesystem with a higher than usual number of inodes (the classic case was an NNTP spool mount).
I literally just solved an issue this weekend that was inode exhaustion: there were years of snapshots and traces laying around and the DBAs were confused as to why the database couldn't create a directory to start and was exiting with "No space left on device" when there was 5% storage free. That lead to the second discussion of why you don't fix it by piping filenames one by one to rm, but either use find's inbuilt…