Earlier quoted context omitted.
I think maybe you should familiarise yourself with how e.g. ext4 works. You can unlink a file and have a process still hold a reference to the inode. This allows you to continue reading (or executing) a file which may not have been fully mapped yet even after its last filesystem reference is gone. There really isn't that much of a good reason to disallow deleting the files (assuming NTFS is capable of supporting a si…
What happens when you run an executable from a FAT formatted partition under Linux, then I would guess Linux also no longer allow deleting the file while it is running, right? In the end this is a feature of the file system, can you delete open files?
It's actually quite intriguing how this works.
When you delete a file on FAT it seems to remove the directory entry, but does not update the free space information and doesn't free the clusters. If you run a fsck.fat on a filesystem in this state, it frees up the space (likewise, if you just let the process exit and unmount the disk normally, this space is also freed).
Presumably the information about where the file exists is only kept in RAM once you delete the directory entry. This should be sufficient to let anyone with an open file handle etc to keep reading it (including the kernel reading additional pages of the executable).