A tale of Phobos – How we almost cracked a ransomware using CUDA • Hacker News Board

Re: A tale of Phobos – How we almost cracked a ransomware using CUDA

#61

Post by jaeckel » Sat, Feb 25, 2023, 9:27 AM UTC

Too wrote: post #60

TacticalCoder wrote: post #27

Earlier quoted context omitted.

I agree. And for some stuff you get cryptographic checksums for free. Backup of Git repositiories: ... # git fsck --full error: unable to unpack contents of .git/objects/a2/cf1a9631658799733f43c3b3f0a799696a4b21 error: a2cf1a9631658799733f43c3b3f0a799696a4b21: object corrupt or missing: .git/objects/a2/cf1a9631658799733f43c3b3f0a799696a4b21 Oops... No matter if it's a malware, the lack of ECC which by bad luck induce…

People normally say don’t store binaries in git. Is this a big issue if the files don’t change very often? From what I understood the biggest problem is they don’t diff well. With photos not changing very often, can it work? Anyone tried using git for 500G of photos? I would love to if it worked, I have my photo collection spread out on multiple computers and merging the edits to the master backup is always a pita. “…

I guess that 500GB repository would be barely usable.

You should check out Git LFS if you want to do that, as it sounds like a good idea in the first place!

Re: A tale of Phobos – How we almost cracked a ransomware using CUDA

#62

Post by zamnos » Sat, Feb 25, 2023, 9:57 AM UTC

Too wrote: post #60

TacticalCoder wrote: post #27

Earlier quoted context omitted.

I agree. And for some stuff you get cryptographic checksums for free. Backup of Git repositiories: ... # git fsck --full error: unable to unpack contents of .git/objects/a2/cf1a9631658799733f43c3b3f0a799696a4b21 error: a2cf1a9631658799733f43c3b3f0a799696a4b21: object corrupt or missing: .git/objects/a2/cf1a9631658799733f43c3b3f0a799696a4b21 Oops... No matter if it's a malware, the lack of ECC which by bad luck induce…

People normally say don’t store binaries in git. Is this a big issue if the files don’t change very often? From what I understood the biggest problem is they don’t diff well. With photos not changing very often, can it work? Anyone tried using git for 500G of photos? I would love to if it worked, I have my photo collection spread out on multiple computers and merging the edits to the master backup is always a pita. “…

Git simply wasn't designed for that and so the key issue with storing binaries in it is what you mentioned last - that the way git works, a full clone has the full history of all the files. Deleting a file in git then doesn't actually delete the file from git history, so a fresh full clone of 500G of photos isn't going to be 500G, it's going to be that, times however many copies exist in history. A shallow clone solves that, and shallow clones supposedly work better these days in latest version of git, but fundamentally you're using a hammer on screws, as it were.

If you're open to new tools, git annex is what you're looking for. The other two options are Subversion, which has some DVCS features these days, or Perforce Helix Core (paid), though I can't vouch for it as I've never used it.

Re: A tale of Phobos – How we almost cracked a ransomware using CUDA

#63

Post by gschoeni » Sat, Feb 25, 2023, 5:32 PM UTC

Too wrote: post #60

TacticalCoder wrote: post #27

Earlier quoted context omitted.

I agree. And for some stuff you get cryptographic checksums for free. Backup of Git repositiories: ... # git fsck --full error: unable to unpack contents of .git/objects/a2/cf1a9631658799733f43c3b3f0a799696a4b21 error: a2cf1a9631658799733f43c3b3f0a799696a4b21: object corrupt or missing: .git/objects/a2/cf1a9631658799733f43c3b3f0a799696a4b21 Oops... No matter if it's a malware, the lack of ECC which by bad luck induce…

People normally say don’t store binaries in git. Is this a big issue if the files don’t change very often? From what I understood the biggest problem is they don’t diff well. With photos not changing very often, can it work? Anyone tried using git for 500G of photos? I would love to if it worked, I have my photo collection spread out on multiple computers and merging the edits to the master backup is always a pita. “…

We've been working on some open source tooling called "oxen" that was built for large datasets of images, video, audio, text etc. We wanted to solve the exact problem you're flagging here with git.

Feel free to check it out here https://github.com/Oxen-AI/oxen-release#-oxen would love any feedback!

Re: A tale of Phobos – How we almost cracked a ransomware using CUDA

#64

Post by jxramos » Mon, Feb 27, 2023, 8:47 PM UTC

TacticalCoder wrote: post #33

jxramos wrote: post #29

Earlier quoted context omitted.

what do you mean adding checksum to the picture, do you add the checksum as a filename suffix eg IMG0001_ .jpg, something like that? Or do you tuck it into the exif data and have a tool that computes the checksum of the file minus the checksum part.

Yup exactly just adding a suffix. I'm not only backing .jpg files. For example I also backup a few screenshots (some are in .png and some are in .webp format). So I don't care about the different pictures (or short family movies) format. I just wrote some Clojure / babashka code to do that. I also truncate the checksum so that the filename doesn't become gigantic: it's not sensitive content, it's just to detect corru…

yah ZFS is supposed to alert somehow, I've been curious about the actual end user experience for that workflow and how it feels. Restoring from backup for disturbed crcs is excellent, I've been hoping to get into that action myself once I discovered various low priority files had bit rot on them.

Re: A tale of Phobos – How we almost cracked a ransomware using CUDA

#65

Post by ganoushoreilly » Wed, Mar 01, 2023, 12:27 AM UTC

ryan-c wrote: post #59

ganoushoreilly wrote: post #51

Earlier quoted context omitted.

There have been tons of projects that tried to repurpose bitcoin hardware. Even if it's not the latest that's a lot of computational power! So far though none of the project I follow have yielded any real results. Here's a interesting read that gives a bit of info on the differences and why it doesn't really work. https://rya.nc/asic-cracking.html

I'm glad people are still finding that post useful.

Thanks for writing it! Most of your posts were interesting reads.