Live data from Hacker News

SD Memory Card Formatter for Linux

sdcard.org

21–30 of 60 posts

Re: SD Memory Card Formatter for Linux

#21
post #19

I've always formatted SD cards like any other block device. This page mentions a secure section of the card that can somehow reduce performance if you don't treat it specially? What is this security section for, and do standard UNIXy partitioning tools have the ability to handle them properly, or do I have to use this proprietary(?) tarball if I want good SD performance?

IIRC there is a reserved area in the beginning of th SD cards flash which is used for internal housekeeping. When this area is overwritten/formatted the card is bricked. [add] The size depends on the size of the card. I think it is around 32KB on cards with a capacity under 2GB.

I do some development with dev boards and firmware stuff and regularly dd images onto SD-card without any skips or something like that. I never bricked any card.

Re: SD Memory Card Formatter for Linux

#23

I've always formatted SD cards like any other block device. This page mentions a secure section of the card that can somehow reduce performance if you don't treat it specially? What is this security section for, and do standard UNIXy partitioning tools have the ability to handle them properly, or do I have to use this proprietary(?) tarball if I want good SD performance?

I don't think the page actually says that the security section has anything to do with the potential for poor performance; those are discussed in separate paragraphs. It's mentioning the security section to warn the user that this tool will not actually wipe every byte of the card, just the normal unsecured data storage area.

The performance pitfalls are not stated, but I'd guess they're referring to the possibility of using suboptimal file system block sizes or alignment. I wouldn't worry too much about recent tools getting alignment wrong or recent cards being overly sensitive to block sizes.

The official SD tool probably also ensures that it uses the exact filesystem called for by their standards (ie. when to use FAT32 and when to use exFAT), as a guarantee of compatibility with any device that interprets the standards too strictly.

Re: SD Memory Card Formatter for Linux

#24
post #19

I've always formatted SD cards like any other block device. This page mentions a secure section of the card that can somehow reduce performance if you don't treat it specially? What is this security section for, and do standard UNIXy partitioning tools have the ability to handle them properly, or do I have to use this proprietary(?) tarball if I want good SD performance?

IIRC there is a reserved area in the beginning of th SD cards flash which is used for internal housekeeping. When this area is overwritten/formatted the card is bricked. [add] The size depends on the size of the card. I think it is around 32KB on cards with a capacity under 2GB.

I'm dubious about this claim, I have completely destroyed the partition table and mkfs on the raw device...

Re: SD Memory Card Formatter for Linux

#25

Earlier quoted context omitted.

It may be proprietary but it's developed by Tuxera, the same lads who created and maintain the NTFS drivers for Linux.

SD Cards have computers inside of them. Security conscious users will be really worried using a blob for their stuff.

Hard drives, USB sticks and SSDs too...

Re: SD Memory Card Formatter for Linux

#26
post #19

I've always formatted SD cards like any other block device. This page mentions a secure section of the card that can somehow reduce performance if you don't treat it specially? What is this security section for, and do standard UNIXy partitioning tools have the ability to handle them properly, or do I have to use this proprietary(?) tarball if I want good SD performance?

IIRC there is a reserved area in the beginning of th SD cards flash which is used for internal housekeeping. When this area is overwritten/formatted the card is bricked. [add] The size depends on the size of the card. I think it is around 32KB on cards with a capacity under 2GB.

You're right but it doesn't matter here as it is fully hidden. Unless you have an SD card with a buggy firmware or access to manufacturer-specific tool, it is not accessible even when you "deep" format it.

Re: SD Memory Card Formatter for Linux

#27
post #9

Earlier quoted context omitted.

It may be proprietary but it's developed by Tuxera, the same lads who created and maintain the NTFS drivers for Linux.

If it is a choice between preparing my sd cards with open partitioning tools, or running some non-auditable official tool in binary form, it is an easy choice. I'll do the former, as I always have. By the way, a cool quote from the agreement preceding the download page: >3. RESTRICTIONS: You agree to NOT: (a) disassemble, reverse engineer, decompile, or otherwise attempt to derive any source code for the SDA Software…

> 3. RESTRICTIONS: You agree to NOT: (a) disassemble, reverse engineer, decompile, or otherwise attempt to derive any source code for the SDA Software from executable code;

Depending on where you live, this might have no legal effect. Some jurisdictions allow explicitly for reverse engineering & co. for different purposes (e.g. research, interoperability with other systems, etc.).

Re: SD Memory Card Formatter for Linux

#28

18 months ago I whined to their devs it'd be nice to have a version of their SD formatter for Linux, et voilà!, 18 months later, BOOM! - Awesome!! ... I"m not sure if my petulant whining and life-draining email verbosity provided the underlying motivation to their acquiescence, but until I'm provided contrary proof I'll assume it's so ;o)

Thank you! Thank the devs more of course ;)

My world depends on writing TBs of data to micro SD cards in difficult environments right at the limit of write speed. These tools are great but our devices are linux (embedded), so this really is a huge help.

For anyone else out there who needs high performance out of an SD card... fstrim is your friend. Seriously, nothing comes close. Run it regularly when not under load.

Re: SD Memory Card Formatter for Linux

#29
I call bullshit, if performance was an issue the companies that make SD cards would have found a way to fix this in the firmware and not have to rely on an external formatting tool when almost every device with an SD slot has the ability to format an SD card.

Re: SD Memory Card Formatter for Linux

#30
So much confusion in this discussion. Let me try to clarify (all of this is very very simplified).

1. NAND in SD cards has page size (minimum writeable piece) something like 4kB or 8KB, or maybe even 64KB. But as per spec SD cards must allow writes at 512 byte granularity. Aligning your file system clusters with these flash pages will allow SIGNIFICANTLY better performance and lower wear. How? Well in the CSD register (accessible using SD commands), the NAND geometry is given, so all it takes is reading that and creating a file system that is properly aligned.

2. Why not just mkfs.vfat? You can, but unless you do some of the above work yourself and some math, you’re unlikely to get the alignment right and will suffer because of it, especially on small or random writes. You CAN do it yourself. Just takes a little work.

3. What about my gopro, my phone, or $DEVICE-X that can format SD cards? Well, most device firmwares DO indeed do this correctly as the SD spec advises to do.

4. Why doesn’t the card just do it in firmware all by itself? Curiously sony memory stick does this. Why not sd? There isn’t a command in the spec for that and adding it now is too late - nobody will use it. Thus we have this tool and ones like it.

5. What is this about the secure area? The tool just says it doesn’t touch it. Almost nobody does. Costs a lot of money to license the SD security stuff. Nobody uses it. It was meant for securing MP3s on cards. Secure area exists on all cards (yes you are paying for flash you cannot use). Luckily it isn’t big. Special commands are used to access it. This tool and basically any device you’ll ever encounter in your everyday life do not use them. Lacking those commands, it is as if the secure area doesn’t exist.

6. Why dont windows/Linux/macOS do it right themselves? My guess? Separation of concerns. The formatting code there only formats a partition. For proper alignment you may need to start the partition on the right block as well. Simple example. Say in your fictional card, we have 4KB pages. That’s 8 512-byte sectors. So we want our first FAT cluster to start at a multiple of 8 (and be a multiple of sectors in size). Say we work out that our various FS Structures need 17 sectors. That means that we need the partition to start at sector 7. Thus we make the MBR do so. Wait! You might say that FAT already has “resevered sectors” value we can use for this. No MBR touching needed. Well, you’d be surprised how many devices/firmwares break with nonstandard values there. Also, what makes you so sure that mkfs.vfat can even find out how many sectors into /dev/sdb, /dev/sdb1 starts, to do the math right? For that matter, what makes you think it would even know to check for SD-card-ness of /dev/sdb?

Post reply on HN