11–20 of 77 posts
No date on the post. Probably old and fixed now. Just checked with a random folder, it happily supported 782292 files.
I think his point still stands. Basically, if you do something far enough outside the norm, you may hit really time consuming bugs.
During my young naïve programming days (which coming to think of it haven't really ended) I thought to myself "Why bother using some third party database when I could just store user information in text files, the OS handles that sort of thing really well, right?". I looked into it as much as I could be bothered and found no huge red flags. After a few years of this server merrily creating a new file for each user I…
During my young naïve programming days (which coming to think of it haven't really ended) I thought to myself "Why bother using some third party database when I could just store user information in text files, the OS handles that sort of thing really well, right?". I looked into it as much as I could be bothered and found no huge red flags. After a few years of this server merrily creating a new file for each user I…
"After a lot of hunting around, I found out that I had hit my inode limit, a concept I was completely unaware of." What was the number you hit? I have a flat file system for something I'm prototyping. The files are in a database with a random SHA256 filename associated with them. I was thinking about doing what Git does with creating folders for parts of the string to alleviate the number of files in one single direc…
During my young naïve programming days (which coming to think of it haven't really ended) I thought to myself "Why bother using some third party database when I could just store user information in text files, the OS handles that sort of thing really well, right?". I looked into it as much as I could be bothered and found no huge red flags. After a few years of this server merrily creating a new file for each user I…
I was hit by this recently - with our 500ish (!!) entries in /node_modules and their dependencies of dependencies of dependencies, a standard 8GB VM formatted with ext4 just simply didn't have enough inodes for us to have two copies of our app (deployment swapover). Had to increase the size to 20GB, just to get the extra inodes. XFS would have handled it much better (from testing on a parallel machine). It was inode exhaustion, though the 'disk full' error message was the same - confusing at first, as df showed 40% free space...
During my young naïve programming days (which coming to think of it haven't really ended) I thought to myself "Why bother using some third party database when I could just store user information in text files, the OS handles that sort of thing really well, right?". I looked into it as much as I could be bothered and found no huge red flags. After a few years of this server merrily creating a new file for each user I…
Another good one that even less people know about is the sticky bit.
Earlier quoted context omitted.
"After a lot of hunting around, I found out that I had hit my inode limit, a concept I was completely unaware of." What was the number you hit? I have a flat file system for something I'm prototyping. The files are in a database with a random SHA256 filename associated with them. I was thinking about doing what Git does with creating folders for parts of the string to alleviate the number of files in one single direc…
It's a global limit per filesystem. You can check what the limits are for your filesysems, and what the current use is with `df -i`. The thing that git does is for speed.
How is that number calculated? It's less than 2^25 but greater than 2^24, so I imagine it's even awkward to store. I guess that has something to do with the file table size for HDD - you can't have more nodes than you have addressable space for those nodes.
I knew Git did it for speed, but I was wondering whether it was also about the number of files that can exist in a single directory.
EDIT: Corrected percentage.
Huh, max 32000 files in a directory seems awfully low. Even crusty old NTFS is supposed to support 4294967295 files. Explorer will not be too happy displaying such a folder though!