“Welcome to the party, pal!”
Spotting base64 encoded JSON, certificates, and private keys
41–50 of 135 posts
Re: Spotting base64 encoded JSON, certificates, and private keys
#42Earlier quoted context omitted.
MII is not RSA, it's an opening header of asn1 structure encoded to DER -- 30 82 0x which is basically "{" when which can be pretty much anything from x509 certificate to private keys fro ECDSA. Actual RSA oid is somewhere in the middle.
True, but for the most part, RSA keys are the only keys that anyone encounters that start with long SEQUENCEs requiring two-byte lengths. `eY` could be any JSON, but it's most likely going to be a JWT. Neither is a perfect signal, but contextually is more likely correct than not.
That depends on the kind of abyss you are staring into. Mine had plenty of non-RSA keys, certificates (which are of course two-byte length all the time) and CMS containers.
Re: Spotting base64 encoded JSON, certificates, and private keys
#43I don't really love this. It just feels so wasteful. JWT does it as well. Even in this example, they are double base64 encoding strings (the salt). It's really too bad that there's really nothing quite like json. Everything speaks it and can write it. It'd be nice if something like protobuf was easier to write and read in a schemeless fashion.
> It'd be nice if something like protobuf was easier to write and read in a schemeless fashion.
If you just want a generic, binary, hierarchical type-length-value encoding, have you considered https://en.wikipedia.org/wiki/Interchange_File_Format ?
It's not that there are widely-supported IFF libraries, per se; but rather that the format is so simple that as long as your language has a byte-array type, you can code a bug-free IFF encoder/decoder in said language about five minutes.
(And this is why there are no generic IFF metaformat libraries, ala JSON or XML libraries; it's "too simple to bother everyone depending on my library with a transitive dependency", so everyone just implements IFF encoding/decoding as part of the parser + generator for their IFF-based concrete file format.)
What's IFF used in? AIFF; RIFF (and therefore WAV, AVI, ANI, and — perhaps surprisingly — WebP); JPEG2000; PNG [with tweaks]...
• There's also a descendant metaformat, the ISO Base Media File Format ("BMFF"), which in turn means that MP4, MOV, and HEIF/HEIC can all be parsed by a generic IFF parser (though you'll miss breaking some per-leaf-chunk metadata fields out from the chunk body if you don't use a BMFF-specific parser.)
• And, as an alternative, there's https://en.wikipedia.org/wiki/Extensible_Binary_Meta_Languag... ("EBML"), which is basically IFF but with varint-encoding of the "type" and "length" parts of TLV (see https://matroska-org.github.io/libebml/specs.html). This is mostly currently used as the metaformat of the Matroska (MKV) format. It's also just complex enough to have a standalone generic codec library (https://github.com/Matroska-Org/libebml).
My personal recommendation, if you have some structured binary data to dump to disk, is to just hand-generate IFF chunks inline in your dump/export/send logic, the same way one would e.g. hand-emit CSV inline in a printf call. Just say "this is an IFF-based format" or put an .iff extension on it or send it as application/x-iff, and an ecosystem should be able to run with that. (And just like with JSON, if you give the IFF chunks descriptive names, people will probably be able to suss out what the chunks "mean" from context, without any kind of schema docs being necessary.)
Re: Spotting base64 encoded JSON, certificates, and private keys
#44[deleted]
Re: Spotting base64 encoded JSON, certificates, and private keys
#45Good knowledge, now explain why it's like that.
{" is ASCII 01111011, 00100010
Base64 takes 3 bytes x 8 bits = 24 bits, groups that 24 bit-sequence into four parts of 6 bits each, and then converts each to a number between 0-63. If there aren't enough bits (we only have 2 bytes = 16 bits, we need 18 bits), pad them with 0. Of course in reality the last 2 bits would be taken from the 3rd character of the JSON string, which is variable.
The first 6 bits are 011110, which in decimal is 30.
The second 6 bits are 110010, which in decimal is 50.
The last 4 bits are 0010. Pad it with 00 and you get 001000, which is 8.
Using an encoding table (https://base64.guru/learn/base64-characters), 30 is e, 50 is y and 8 is I. There's your "ey".
Funny how CS people are so incurious now, this blog post touches the surface but didn't get into the explanation.
Re: Spotting base64 encoded JSON, certificates, and private keys
#46I don't really love this. It just feels so wasteful. JWT does it as well. Even in this example, they are double base64 encoding strings (the salt). It's really too bad that there's really nothing quite like json. Everything speaks it and can write it. It'd be nice if something like protobuf was easier to write and read in a schemeless fashion.
What’s wrong with this? The purpose of Base64 is to encode data—especially binary data—into a limited set of ASCII characters to allow transmission over text-based protocols. It is not a cryptographic library nor an obfuscation tool. Avoid encoding sensitive data using Base64 or include sensitive data in your JWT payload unless it is encrypted first.
JSON is already text based and not binary so encoding it with base64 is bit wasteful. Especially if you are going to just embed the text in another json document.
And of course text-based things themselves are quite wasteful.
Re: Spotting base64 encoded JSON, certificates, and private keys
#47Something similar pops up if you have to spend a lot of time looking at binary blobs with a hex editor. Certain common character sequences become familiar. This also leads to choosing magic numbers in data formats that decode to easily recognized ASCII strings. I'm sure if I worked with base64 I'd be choosing something that encoded nicely into particular strings for the same purpose.
Related trick I've learnt: binary data containing lots of 0x40 may be EBCDIC text, or binary data containing embedded EBCDIC strings – 0x40 is EBCDIC space character
Probably not a very useful trick outside of certain specific environments
Re: Spotting base64 encoded JSON, certificates, and private keys
#48I debugged way too many JWT tokens
I know eyJhbG by heart
Re: Spotting base64 encoded JSON, certificates, and private keys
#49I don't really love this. It just feels so wasteful. JWT does it as well. Even in this example, they are double base64 encoding strings (the salt). It's really too bad that there's really nothing quite like json. Everything speaks it and can write it. It'd be nice if something like protobuf was easier to write and read in a schemeless fashion.
What’s wrong with this? The purpose of Base64 is to encode data—especially binary data—into a limited set of ASCII characters to allow transmission over text-based protocols. It is not a cryptographic library nor an obfuscation tool. Avoid encoding sensitive data using Base64 or include sensitive data in your JWT payload unless it is encrypted first.
Exactly. Using base64 as an obfuscation tool, or (shudder) encryption is seriously misusing it for what it was originally intended for. If that's what you need to do then avoid using base64 in favor for something that was designed to do that.
Re: Spotting base64 encoded JSON, certificates, and private keys
#50I built a JWT support library at work ( https://github.com/geldata/gel-rust/tree/master/gel-jwt ) and I can confirm that JWTs all sound like "eyyyyyy" in my head.
"Is that you again, punk zip?" when seeing the first few bytes of a zip file.