21–30 of 77 posts
No date on the post. Probably old and fixed now. Just checked with a random folder, it happily supported 782292 files.
NB: My current working folders are now in Google Drive, but is essentially the same folder structure as I started on our Novell server about 25 years ago...
Huh, max 32000 files in a directory seems awfully low. Even crusty old NTFS is supposed to support 4294967295 files. Explorer will not be too happy displaying such a folder though!
I came to the same conclusion about sharding files from the get go. I had a similar experience where my host wouldn't accept to synchronize a directory with more than 5000 files in it. (Software producing the files is my own note-taking application where each page is stored as an individual file to speed up backup of only the pages that changes).
Earlier quoted context omitted.
It's a global limit per filesystem. You can check what the limits are for your filesysems, and what the current use is with `df -i`. The thing that git does is for speed.
I've apparently used 1241165 out of 19005440, or about 6.53% on my development machine. It's not inconceivable to have 19 million files, though, if you had one per user. That's not great. How is that number calculated? It's less than 2^25 but greater than 2^24, so I imagine it's even awkward to store. I guess that has something to do with the file table size for HDD - you can't have more nodes than you have addressab…
During my young naïve programming days (which coming to think of it haven't really ended) I thought to myself "Why bother using some third party database when I could just store user information in text files, the OS handles that sort of thing really well, right?". I looked into it as much as I could be bothered and found no huge red flags. After a few years of this server merrily creating a new file for each user I…
It was much easier than coding a database back-end, with no need to run a database server. And, when we needed to scale beyond one machine, it was trivial to use a central NFS server (read-only NFS worked really well for this) and multiple front-end web servers.
Though we did know how many files we'd be hosting, so we sized the number of bytes per inode appropriately.
During my young naïve programming days (which coming to think of it haven't really ended) I thought to myself "Why bother using some third party database when I could just store user information in text files, the OS handles that sort of thing really well, right?". I looked into it as much as I could be bothered and found no huge red flags. After a few years of this server merrily creating a new file for each user I…
Btrfs?
Brings back memories of the old DOS days, when there was quite a low limit on the number of files that you could store in the root directory. I can't remember the exact numbers, but it lead to my early habits of creating a new sub directory for every little thing so nowadays (over 3 decades later) nearly everything I need is at least 5 levels deep! ;) NB: My current working folders are now in Google Drive, but is ess…
512?
https://en.wikipedia.org/wiki/File_Allocation_Table#Final_FA... (see the last paragraph.)
"If you're working with huge files or a huge number of files, use XFS" went the slogan. I was hit by this recently - with our 500ish (!!) entries in /node_modules and their dependencies of dependencies of dependencies, a standard 8GB VM formatted with ext4 just simply didn't have enough inodes for us to have two copies of our app (deployment swapover). Had to increase the size to 20GB, just to get the extra inodes. X…
Brings back memories of the old DOS days, when there was quite a low limit on the number of files that you could store in the root directory. I can't remember the exact numbers, but it lead to my early habits of creating a new sub directory for every little thing so nowadays (over 3 decades later) nearly everything I need is at least 5 levels deep! ;) NB: My current working folders are now in Google Drive, but is ess…
> when there was quite a low limit on the number of files that you could store in the root directory. I can't remember the exact numbers 512? https://en.wikipedia.org/wiki/File_Allocation_Table#Final_FA... (see the last paragraph.)