Static analysis of an unknown compression format
blog.lse.epita.fr
Static analysis of an unknown compression format
1–10 of 14 posts
Re: Static analysis of an unknown compression format
#2Is there any tool to generate bitmaps from binary files? For example using each byte as grey value and letting the user specify the column width. Or with more than one byte and then using color.
Wouldn't this be a trivial and somewhat useful thing to see structure in binary files?
Re: Static analysis of an unknown compression format
#3Is there any tool to generate bitmaps from binary files? For example using each byte as grey value and letting the user specify the column width. Or with more than one byte and then using color. Wouldn't this be a trivial and somewhat useful thing to see structure in binary files?
[deleted]
Re: Static analysis of an unknown compression format
#4Is there any tool to generate bitmaps from binary files? For example using each byte as grey value and letting the user specify the column width. Or with more than one byte and then using color. Wouldn't this be a trivial and somewhat useful thing to see structure in binary files?
When doing simple graphics work at university, I would name files that contained nothing but pixel data foo.raw and IIRC some image editors would offer the options you mention when opening the files, then display them fine. Maybe that was before .raw got associated with cameras (and presumably more complex).
Re: Static analysis of an unknown compression format
#5Is there any tool to generate bitmaps from binary files? For example using each byte as grey value and letting the user specify the column width. Or with more than one byte and then using color. Wouldn't this be a trivial and somewhat useful thing to see structure in binary files?
When doing simple graphics work at university, I would name files that contained nothing but pixel data foo.raw and IIRC some image editors would offer the options you mention when opening the files, then display them fine. Maybe that was before .raw got associated with cameras (and presumably more complex).
raw (can be) interpreted from any stream of binary data, you can open EXEs in photoshop as RAW, save it as a PNG, transfer it, then resave it as a raw and it will run.
Re: Static analysis of an unknown compression format
#6Is there any tool to generate bitmaps from binary files? For example using each byte as grey value and letting the user specify the column width. Or with more than one byte and then using color. Wouldn't this be a trivial and somewhat useful thing to see structure in binary files?
In my favorite static analysis toolset (Radare, http://radare.org/), there's a nice feature similar to this. It "zooms" out on a big file by summarizing portions of it into chunks and displaying colored ascii characters in a chart to represent the block.
With the entropy and ASCII printable filters setup, it makes repeating patterns and TEXT sections pretty clear.
Re: Static analysis of an unknown compression format
#7Is there any tool to generate bitmaps from binary files? For example using each byte as grey value and letting the user specify the column width. Or with more than one byte and then using color. Wouldn't this be a trivial and somewhat useful thing to see structure in binary files?
You might find this useful: http://corte.si/posts/visualisation/binvis/index.html
and related: http://corte.si/posts/visualisation/entropy/index.html (visualizing entropy in binary files)
I used his software on an unknown format from the 1988 game Circuit's Edge to find out if/where the file was encrypted or compressed. It works well and was able to give me a (very) general idea of where in the file I needed to look further.
Edit: Nope, I remembered incorrectly. I used it on the game's .exe to tell me if I was dealing with a compressed executable. It turned out that it was compressed and so I went on a search for an ancient unpacker.
See before and after:
edge.exe packed: http://imgur.com/PlHGl
edge.exe unpacked: http://imgur.com/jeFLb
Re: Static analysis of an unknown compression format
#8Is there any tool to generate bitmaps from binary files? For example using each byte as grey value and letting the user specify the column width. Or with more than one byte and then using color. Wouldn't this be a trivial and somewhat useful thing to see structure in binary files?
Something I wrote a while back, using Python and PIL.
import sys
import math
from PIL import Image
def getSize(len, width):
h = int(math.ceil(float(len) / width))
return (width, h)
def getImage(file, width=256):
d = file.read()
s = getSize(len(d), width)
im = Image.new('L', s)
im.putdata(d)
return im
def main():
if len(sys.argv) [width]'
return
with open(sys.argv[1], 'rb') as f:
if len(sys.argv) > 3:
width = int(sys.argv[3])
img = getImage(f, width)
else:
img = getImage(f)
img.save(sys.argv[2])
if __name__ == "__main__":
main()
Re: Static analysis of an unknown compression format
#9Reminds me of reverse engineering an email archiving format used by EMC IIRC. It used an ancient PKZIP compression algorithm to compress COM IStream data and then wrapped the compressed data in what was essentially a proprietary linked list.
Re: Static analysis of an unknown compression format
#10Is there any tool to generate bitmaps from binary files? For example using each byte as grey value and letting the user specify the column width. Or with more than one byte and then using color. Wouldn't this be a trivial and somewhat useful thing to see structure in binary files?
It is very simple to create the binary versions of http://en.wikipedia.org/wiki/Netpbm_format from the command line, if you have that binary chunk in a file.
Something like:
echo P4 > file
echo "640 480" >> file
echo 255 >> file
cat binaryBlob >> file