Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BigFAT – Backward compatible FAT extension for unlimited file size

segger.com

Post reply on HN
Print view View original

81–90 of 109 posts

Re: BigFAT – Backward compatible FAT extension for unlimited file size

#81

Post by mort96 »

DannyBee wrote: post #70
mort96 wrote: post #10

I like the idea. Making it backwards compatible with FAT means that, in principle, regular FAT filesystem implementations could be transparently changed to support big fat files (hehe) transparently. However, reading the spec, it doesn't look fully backwards compatible? It seems like there are file structures which are possible to represent in FAT which aren't possible to represent in BigFAT. In FAT, I could have a 4…

It's as backwards compatible as any other fat extension done so far. For example, LFN fails if you create too many files with the same first 6 letters :) I'm actually honestly not sure why representing all legal FAT32 file structures is a particularly useful goal? FAT in particular, in all of it's forms, has always had limitations and weirdness in filenames, etc.

I don't understand your LFN example. Which FAT file structure can be represented with LFN disabled that's no longer possible to represent if you add support for LFN?

If BigFAT was actually backwards compatible, it would've been a no-brainer to add support for in filesystem drivers. But since it changes the interpretation of some legitimate structures, adding support for BigFAT is a breaking change. I don't know whether operating systems will want to make breaking changes to their FAT32 filesystems, but it certainly seems like a bigger ask.

Re: BigFAT – Backward compatible FAT extension for unlimited file size

#82

Post by mort96 »

NelsonMinar wrote: post #43
mort96 wrote: post #10

I like the idea. Making it backwards compatible with FAT means that, in principle, regular FAT filesystem implementations could be transparently changed to support big fat files (hehe) transparently. However, reading the spec, it doesn't look fully backwards compatible? It seems like there are file structures which are possible to represent in FAT which aren't possible to represent in BigFAT. In FAT, I could have a 4…

What FAT32 filesystem in the real world has a file named "foo.000.BigFAT" on it?

See my response to https://news.ycombinator.com/item?id=32753207. I'm not saying it will break everyone's FAT32 drives, but it is a breaking change in a filesystem, which seems like something kernel people would usually try to avoid.

Re: BigFAT – Backward compatible FAT extension for unlimited file size

#83

Post by vetinari »

xxpor wrote: post #57
vetinari wrote: post #39

Earlier quoted context omitted.

Yes, it does. If you manufacture 100K of them, and save 10 dollars on every piece, you got an extra million in savings.

From the wiki, my understanding is the licence is $0.25 a unit, not $10?

It was a number to illustrate a point, not exact figure for this specific case.

Yes, the manufacturers will go great lengths to minimize variable costs. If they can shave $0.25, they will. At volumes, it matters.

Re: BigFAT – Backward compatible FAT extension for unlimited file size

#84

Post by mort96 »

jonny_eh wrote: post #75
londons_explore wrote: post #74

Earlier quoted context omitted.

I can imagine that if bigfat is successful, such files will start to exist. Imagine someone takes a bigfat drive and puts it in a non-bigfat capable machine, then zips up a directory and publishes it. When that directory is unzipped on a bigfat machine, should the bigfat files be re-joined, or should they show as separate files? One breaks the OS file API and the unzip program might crash/fail, while the other leads…

> should the bigfat files be re-joined, or should they show as separate files They're only "rejoined" by the BigFAT compatible filesystem driver on access. By running such a driver, you're agreeing that such files should "appear" as one.

Hey, notice that you're suggesting that BigFAT should be disabled by default here; you think the user should have to choose to be running a driver with BigFAT-support. Maybe reflect on whether that's a desirable situation, or if it would've been preferable if the feature could've been enabled by default.

Re: BigFAT – Backward compatible FAT extension for unlimited file size

#85

Post by rleigh »

winter_blue wrote: post #65
selfhoster11 wrote: post #62

Earlier quoted context omitted.

BtrFS is unsafe for production use unless it's coupled with really good backups. On the other hand, NTFS on Windows, Ext* on Linux, or ZFS on any supported OS, has not been known to eat data as frequently.

Is BtrFS without RAID safe? According to them ( https://btrfs.wiki.kernel.org/index.php/Status ) only RAID56 is unstable.

There are still major bugs in the rest of it. You can trivially corrupt a mirror. There are examples for how to reproduce it exactly in qemu with virtual drives. I caused me total dataloss on my first Btrfs filesystem at least 8 years back. That bug is apparently still there. The unbalancing issues are still there. I have zero trust of Btrfs in any form.

Re: BigFAT – Backward compatible FAT extension for unlimited file size

#86

Post by chasil »

selfhoster11 wrote: post #62
chasil wrote: post #17

Earlier quoted context omitted.

There are so many good filesystems out there. Is it really necessary to keep dragging FAT along? ReactOS is using btrfs, which has so many useful options that FAT will never see (zstd, xxhash, flash-aware options, snapshots, send/receive, etc.). This is positioned both for Linux and Windows. Microsoft itself restrains ReFS to enterprise use, and btrfs offers so much more functionality. We should stop using a file sys…

BtrFS is unsafe for production use unless it's coupled with really good backups. On the other hand, NTFS on Windows, Ext* on Linux, or ZFS on any supported OS, has not been known to eat data as frequently.

With all due respect, this is now far from true.

"So, we'll repeat this once more: as a single-disk filesystem, btrfs has been stable and for the most part performant for years."

https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu...

Re: BigFAT – Backward compatible FAT extension for unlimited file size

#87

Post by teknopaul »

guipsp wrote: post #51
chasil wrote: post #48

Earlier quoted context omitted.

Microsoft just wants a check from you. We are all forced to pay for this ancient software every time we buy a device that uses it. Wouldn't this money be better used elsewhere? https://en.m.wikipedia.org/wiki/File_Allocation_Table#Patent...

Microsoft isn't getting a check from me using FAT32.

Hmm that's like saying Google is free.

Re: BigFAT – Backward compatible FAT extension for unlimited file size

#88

Post by pantalaimon »

bArray wrote: post #63

Is this only compatible with FAT32, or is it also compatible with FAT12/16? It would be very cool if this would support floppy disks. Regarding the format, once you convert it, does the target device need to have a driver to support the format? It mentions that this would allow for > 4GB files for TVs, but these are typically non-updated very out of date OSes. I think MS missed a trick by not making the boot sector a…

How do you get files > 4 GiB on a floppy disk?

Re: BigFAT – Backward compatible FAT extension for unlimited file size

#89

Post by lproven »

mort96 wrote: post #81
DannyBee wrote: post #70

Earlier quoted context omitted.

It's as backwards compatible as any other fat extension done so far. For example, LFN fails if you create too many files with the same first 6 letters :) I'm actually honestly not sure why representing all legal FAT32 file structures is a particularly useful goal? FAT in particular, in all of it's forms, has always had limitations and weirdness in filenames, etc.

I don't understand your LFN example. Which FAT file structure can be represented with LFN disabled that's no longer possible to represent if you add support for LFN? If BigFAT was actually backwards compatible, it would've been a no-brainer to add support for in filesystem drivers. But since it changes the interpretation of some legitimate structures, adding support for BigFAT is a breaking change. I don't know wheth…

My favourite sobriquet for MS has long been the DOS view of an Office 97 installation in the filesystem:

MICROS~1

(Long name, Microsoft Office.)

So if you create files called MICROS~2 - MICROS~0 in theory you can create enough abbreviated names that there are not available short names for long filenames you wish to create. Every LFN must have a "real" 8.3 counterpart.

Re: BigFAT – Backward compatible FAT extension for unlimited file size

#90

Post by axiolite »

mort96 wrote: post #19
chasil wrote: post #17

Earlier quoted context omitted.

There are so many good filesystems out there. Is it really necessary to keep dragging FAT along? ReactOS is using btrfs, which has so many useful options that FAT will never see (zstd, xxhash, flash-aware options, snapshots, send/receive, etc.). This is positioned both for Linux and Windows. Microsoft itself restrains ReFS to enterprise use, and btrfs offers so much more functionality. We should stop using a file sys…

Camera manufacturers and SD card manufacturers can't start shipping SD cards formatted with btrfs until Windows supports it out of the box. They can start shipping SD cards formatted with FAT32 and software/firmware which reads and writes FAT32+BigFAT.

> can't start shipping SD cards formatted with btrfs until Windows supports it out of the box

3rd parties can write drivers for Windows, you know. A small, read-only FAT partition on a USB stick or SD card could contain the installable drivers necessary to read/write the rest of the disk.

However, that's unnecessary. The best option for a universal file system is UDF. Windows, Mac, and Linux all have full read/write support.

See: https://github.com/JElchison/format-udf

Post reply on HN