Earlier quoted context omitted.
I don't think there are any guarantees here, but if he kept on saving the same file with the same name, it probably overwrote the same sector on disk.
I guess it depends on the text editor. Some editors will save new versions of a file with the same name by writing the current buffer to a new file and then atomically moving that file to the target location, which has no reason to write on the same sectors.
Recovering deleted files using only grep
31–40 of 46 posts
Re: Recovering deleted files using only grep
#32Yeah, been there, done that, although my first step is to umount and dd onto another volume so the sectors don't get reallocated and obliterate the data - and that's generally a sane first step for ANY recovery of this ilk. You haven't lived until you unwittingly run rm -rf on a nfs mount of / on a remote box. Which happens to be the fileserver for a trading shop. In the middle of trading hours.
I seem to remember having a problem where I'd rm-ed a file I needed and using lsof to find the file handle and then being able to cat the data in to a new file using the handle instead of the filename? Details pretty fuzzy, sorry.
Edit:
Example of recovery this way, tested now, works for me - http://pastebin.com/c2djEcqr - the crucial part that I was forgetting is that there needs to be a file handle somewhere that's still open, which is probably not true in most cases. Worth a quick check before unmounting.
I've used ddrescue and photorec for these sort of "issues" before with much success.
Re: Recovering deleted files using only grep
#33Earlier quoted context omitted.
I guess it depends on the text editor. Some editors will save new versions of a file with the same name by writing the current buffer to a new file and then atomically moving that file to the target location, which has no reason to write on the same sectors.
What is the reason for saving then moving for existing files? This an optimization specific to certain file systems?
Re: Recovering deleted files using only grep
#34My editor was still open, and the gocode code-completion daemon too. I looked up some way to dump a process's memory from StackOverflow and used it. (I used a Python script[1], but I see gdb's gcore command and other ways recommended elsewhere.) It worked out. An extra complication was that my home directory is encrypted, so searching the raw disk was out.
So: never too early to use version control. Also, if you think you even might have fat-fingered something, think a second before you do anything that could make the situation worse. I didn't pay enough attention to the error message that could have told me I'd messed up, and answered 'yes' to my editor asking if I should close the file, thinking it had just moved--if I hadn't done that it'd've saved a lot of time. On the other hand, had I kept on charging forth once I realized I had messed up, I could've easily closed the editor, ending the gocode process I wound up recovering stuff from.
So, yeah: be smarter than I was, y'all.
[1] http://unix.stackexchange.com/questions/6267/how-to-re-load-...
Re: Recovering deleted files using only grep
#35Yeah, been there, done that, although my first step is to umount and dd onto another volume so the sectors don't get reallocated and obliterate the data - and that's generally a sane first step for ANY recovery of this ilk. You haven't lived until you unwittingly run rm -rf on a nfs mount of / on a remote box. Which happens to be the fileserver for a trading shop. In the middle of trading hours.
Indeed. The whole "never run rm -rf " thing is so ingrained that someone I know took real joy in finally getting to legitimately use this to nuke an old server once the new one was signed off. Pity he'd not considered that the old one had nfs mounts to the live data...
Re: Recovering deleted files using only grep
#36Yeah, been there, done that, although my first step is to umount and dd onto another volume so the sectors don't get reallocated and obliterate the data - and that's generally a sane first step for ANY recovery of this ilk. You haven't lived until you unwittingly run rm -rf on a nfs mount of / on a remote box. Which happens to be the fileserver for a trading shop. In the middle of trading hours.
I was sysadmin for a small ISP about 15 years ago. I was on vacation about 1500 miles away when I called to check in, the owner gets on the phone and says, "I deleted the entire /bin directory on [the primary web server], is that bad?" I told him to not touch it til I got home, and whatever he did do NOT shut it off! Thankfully we had two machines which were virtually identical OS-wise (RedHat 6 if memory serves). I…
The extent to which new calls to deleted files are made will have a strong impact on this.
Re: Recovering deleted files using only grep
#37My success rate in recovery was markedly better than Navigator's capability in running without crapping out.
The most painful story I've hear was of a BSD admin who had to recover gzipped financial audit files from a corrupted disk, requiring both recovery from media and reconstructed files from fragments of the compressed files. Apparently somewhat painful, but not entirely without success.
Re: Recovering deleted files using only grep
#38Earlier quoted context omitted.
It's kind of flawed where he says " If you pick x to be big enough, you should get the entire file, plus a bit of junk around the edges." - because this is simply not how filesystems work - if you manage to get the full file, it's by luck that it was small enough for the filesystem to allocate its contents contiguously on the block device. Anything bigger and you're gonna quickly need something more sophisticated whi…
I'm the author of the post -- that's good to know, thanks! :) I know embarrassingly little about filesystems. I'm glad you pointed this out. EDIT: though, I'd point out that if you really wanted to recover the file you should probably try to use /proc or something (at the time I didn't know about this). This approach requires crawling the disk which is obv pretty slow. :) It's less of a "here's a useful thing" and mo…
You can simply copy the (proc) file to a new location to recover it.
Remember: open files keep the contents present on disk until the controlling process exists or closes the filehandle.
Since you're actually accessing the file on disk your issues of storage contingency don't come into play -- it's all read back to you in proper file order.
But yes, files (and virtual files) on Linux are pretty slick.
I also remember being really excited learning about disk image files and the ways in which they can be manipulated. Including the options of creating filesystems on virtual disks, partitioning them, and them mounting those partitions, etc. First tried with bootable floppy distros, but I've played around with them in a bunch of contexts since.
Re: Recovering deleted files using only grep
#39Earlier quoted context omitted.
I guess it depends on the text editor. Some editors will save new versions of a file with the same name by writing the current buffer to a new file and then atomically moving that file to the target location, which has no reason to write on the same sectors.
What is the reason for saving then moving for existing files? This an optimization specific to certain file systems?
If you don't do this and you're overwriting a file directly and the write fails for some reason, the data from the old file will be gone and you'll only have a partially-written new file in its place.
This also helps with systems that continuously poll files and watch for changes. If you have, say, a compiler watching your file, you don't want it to start compiling a partially-written version of your file and give you some strange error just because it happened to poll before the write finished.
Re: Recovering deleted files using only grep
#40I guess it's a good thing he didn't have an SSD and TRIM enabled!
Not sure why you are getting down-voted. I assume you are referencing the SSD forensics article that was on here a while back that brought up the fact that TRIM on an SSD can cause forensic recovery issues. Basically blocks can be cleared even if the drive is unmounted but powered on. SSD designers developed an interface allowing the operating system (e.g. Windows, Linux, Mac OS X etc.) to inform the controller that…
When you delete a file on a mechanical hard drive the physical contents of the file still exist on disk, so you can use tricks like these to recover deleted data.
When the drive is then told to write over these locations it doesn't matter that there is old data there and it writes the new data to the location.
SSDs however store data in pages, and while they can write directly to an empty page they can not write directly to a page that already has data in it. Instead, an SSD has to read the current data from the page, modify that data with the new data that it wants to be there, and write the new data to the whole page at once. This is called a read-modify-write operation and is a major reason why SSDs (even now) decrease in performance as they fill up.
The issue is that when you delete a file on disk there is no way for the SSD to know that those data blocks aren't important anymore (without TRIM). The controller of the SSD has to manage a full drive of data (even if you're only actually using some percent of it) and only figures out that a file was deleted when it is finally told to write something else to that location.
TRIM tells the SSD that a file was removed and allows a controller to recover that area to help maintain its performance.
There is a really good discussion of this topic in this article from way back in 2009: http://www.anandtech.com/show/2829