Spoiler: half the time, that unknown file format is a zip containing a sqlite db
Reverse Engineering Unknown File Formats with ImHex
11–20 of 55 posts
Re: Reverse Engineering Unknown File Formats with ImHex
#12I’m going to try that on the newest outlook binary format used to store the emails locally... why couldn’t they stick to the plain simple sqlite?
Actually one of the worst and most insane file formats of all time. I partially reverse engineered it a while ago. My parser is super janky and I can't share it, but I pasted the code into claude and had it summarize it: https://gist.github.com/anchpop/a14325cc451b04a5bf78c476ac20...
Re: Reverse Engineering Unknown File Formats with ImHex
#13I tried to do a similar thing last year to datamine a gacha game's assets and somehow did it without knowing the existence of tools like IDA/Ghidra. They're using a custom db format that was XORed with a key. Iirc it was a bizarre format, it's actually an archive where each column is a file containing all the rows of that column. So a table is just a folder full of these files. Maybe it's a common thing idk. I got a partially working XOR key so I used ImHex to look at the partially decrypted db file and manually looked at the decrypted hex values to "correct" them.
I wish I read something like this article at that time!
Re: Reverse Engineering Unknown File Formats with ImHex
#14Re: Reverse Engineering Unknown File Formats with ImHex
#15Story time: I worked the night shift at a motel during college. "Bored to tears" was an understatement. I was poking around the front desk computer system one night, as one does, and found its data file. Lacking anything else to do, I wrote a little hex dumper in BASIC so I could explore the file. The first thing I noted was that customer names were spaced exactly N bytes apart. Oooh, fixed records! Then I spent the…
How did your program suggest auto-fixed names? A manually entered list of common names, or did it go by previous records?
BASIC is over 40 years older than me, and I have never played with it, thus my question :-)
Re: Reverse Engineering Unknown File Formats with ImHex
#16ImHex makes tackling proprietary formats so much less painful. Seriously a lifesaver for quickly mapping out structures.
Re: Reverse Engineering Unknown File Formats with ImHex
#17I tried to do a similar thing last year to datamine a gacha game's assets and somehow did it without knowing the existence of tools like IDA/Ghidra. They're using a custom db format that was XORed with a key. Iirc it was a bizarre format, it's actually an archive where each column is a file containing all the rows of that column. So a table is just a folder full of these files. Maybe it's a common thing idk. I got a…
like apache parquet?
Re: Reverse Engineering Unknown File Formats with ImHex
#18Spoiler: half the time, that unknown file format is a zip containing a sqlite db
for games its more often .net BinarySerializer.
Re: Reverse Engineering Unknown File Formats with ImHex
#19I finally dipped into using ImHex when doing some reversing of the PS2 memory card format for https://ps2iodb.com.
The program is so good, and has enough of a reputation that I knew to reach for it even though I know nothing about reversing, and that was 3-4 years ago now! Also shoutout to the awesome GUI library is uses; https://github.com/ocornut/imgui
Re: Reverse Engineering Unknown File Formats with ImHex
#20Story time: I worked the night shift at a motel during college. "Bored to tears" was an understatement. I was poking around the front desk computer system one night, as one does, and found its data file. Lacking anything else to do, I wrote a little hex dumper in BASIC so I could explore the file. The first thing I noted was that customer names were spaced exactly N bytes apart. Oooh, fixed records! Then I spent the…
I did something just like this at my first post-college job in the 90s. I found some weird backup files on the NT LAN, had similar observations as you, but wrote my extractor in C (casting records into structs - my first C program) using a “bcc” floppy disk. Turned out to be the entire insurance company Btrieve database, which I could dump into csv to load into MS Access.
This “database” allowed me to automate nearly my entire job - finding and exercising test cases for policy bug reports with a calculator - which gave me time to experiment with software development. I’d gone to college to be a writer, but this was waaaay cooler.
I didn’t get a bonus - heck I’d have been fired for sure - but that time spent learning made my entire career possible. Hoping to get another ten years before AI eats it all.