[dead]
Reverse Engineering Unknown File Formats with ImHex
21–30 of 55 posts
Re: Reverse Engineering Unknown File Formats with ImHex
#22Re: Reverse Engineering Unknown File Formats with ImHex
#23I’ve had a lot of luck with ImHex looking at various binary formats and even using it to aid in writing some file magic. It would be really nice to be able to roundtrip C headers with it. The syntax is close, but often I older formats there is a 1:1 mapping with C structs that doesn’t quite match the imhex syntax.
I use both ImHex and Kaitai struct, give it a try maybe you'll like it
Re: Reverse Engineering Unknown File Formats with ImHex
#24My few rules are:
1) What's the entropy? This helps with encrypted/compressed. You probably need to overcome this?
2) What's the context? Apps are developed in context and context suggests formats. Is it C? Then expect structs. Does it need to go over the wire? Expect run-length encodings. Python/JS - JSON/pickle.
3) There are broadly speaking only a few ways you can read data back into memory: fixed layout, run-length encoding, terminators. A great case-study is ASN.1, once you've built an ASN.1 parser by hand no file format will ever daunt you.
Re: Reverse Engineering Unknown File Formats with ImHex
#25I'm always on the lookout for a good hex editor. I've tried ImHex a few times and it doesn't quite fit the need I have and I prefer HxD for the simple things. The Pattern Language is pretty neat, I hadn't looked into that before. For the complicated things, it's not always a linear process like in the case of the blog article. Mainly, there's often lots of gaps in my structure until I figure them out. Some tools like ImHex or Kaitai, you can put in placeholder fields between two known objects, but that's kind of annoying. 010 Editor lets you highlight and put colored bookmarks, which kind of works. The workflow I've ended up using for reversing in-memory data structures is to get a hexdump of the memory, take a screenshot and then put it in OneNote. Then I can draw colored boxes or highlight and put annotations off on the side with what things are or notes. If a field is a memory address to another related and unknown structure, I often will then get a short dump of that memory and put a screenshot of it on the same page with an arrow to it. It's pretty kludgy, but I like the freeform nature of it. It's basically a RE notebook, just without any useful RE built-ins. If anyone has ideas or recommendations on better discovery workflows, I'd love to hear them.
Re: Reverse Engineering Unknown File Formats with ImHex
#26- stupid question: do you have any ideas how to go about doing this on a .unr file
- it comes from a game called splinter cell conviction made in custom unreal 2.5
- I have neither been able to get UE-Explorer or UEViewer to work with it
- I can send you the file, I just want to know how to change enemy AI spawn types on it
Re: Reverse Engineering Unknown File Formats with ImHex
#27- stupid question: do you have any ideas how to go about doing this on a .unr file - it comes from a game called splinter cell conviction made in custom unreal 2.5 - I have neither been able to get UE-Explorer or UEViewer to work with it - I can send you the file, I just want to know how to change enemy AI spawn types on it
Try an LLM, either to do it directly or to guide your own explorations
Re: Reverse Engineering Unknown File Formats with ImHex
#28One trick that saves a lot of time before you write any pattern: generate two saves that differ in exactly one known way (buy one item, then diff), and byte-diff them. The changed offsets are usually the field you care about, and the size of the changed run tells you the width. Doing this a few times gives you a rough field map without guessing at structure.
The other thing worth checking early is whether the file is compressed or checksummed. If entropy is flat and high across the whole file it is probably deflate or an encrypted blob, and hex staring will get you nowhere until you unwrap it. And if there is a 4 byte value near the header or footer that changes on every save even when the data is identical, treat it as a CRC or timestamp before assuming it is data. Writing back an edited file that fails the checksum is the classic reason a patched save silently refuses to load.
Re: Reverse Engineering Unknown File Formats with ImHex
#29Love imhex and recognised the author straight away. Used it plenty of times got save file reverse engineering to develop save editors
Re: Reverse Engineering Unknown File Formats with ImHex
#30Story time: I worked the night shift at a motel during college. "Bored to tears" was an understatement. I was poking around the front desk computer system one night, as one does, and found its data file. Lacking anything else to do, I wrote a little hex dumper in BASIC so I could explore the file. The first thing I noted was that customer names were spaced exactly N bytes apart. Oooh, fixed records! Then I spent the…
I did something just like this at my first post-college job in the 90s. I found some weird backup files on the NT LAN, had similar observations as you, but wrote my extractor in C (casting records into structs - my first C program) using a “bcc” floppy disk. Turned out to be the entire insurance company Btrieve database, which I could dump into csv to load into MS Access. This “database” allowed me to automate nearly…
Another ten years - really optimistic! :)