I debugged way too many JWT tokens I know eyJhbG by heart
they technically don't need to begin like that! JWT is JSON and is therefore infamously vague... but in practice they for some reason always begin with "alg" so always like eyJhbG
Spotting base64 encoded JSON, certificates, and private keys
51–60 of 135 posts
Re: Spotting base64 encoded JSON, certificates, and private keys
#52I don't really love this. It just feels so wasteful. JWT does it as well. Even in this example, they are double base64 encoding strings (the salt). It's really too bad that there's really nothing quite like json. Everything speaks it and can write it. It'd be nice if something like protobuf was easier to write and read in a schemeless fashion.
What’s wrong with this? The purpose of Base64 is to encode data—especially binary data—into a limited set of ASCII characters to allow transmission over text-based protocols. It is not a cryptographic library nor an obfuscation tool. Avoid encoding sensitive data using Base64 or include sensitive data in your JWT payload unless it is encrypted first.
I think it's more the waste of space in it all. Encoding data in base64 increases the length by 33%. So base64-encoding twice will blow it up by 33% of the original data and then again 33% of the encoded data, making 69% in total. And that's before adding JSON to the mix...
And before "space is cheap": JWT is used in contexts where space is generally not cheap, such as in HTTP headers.
Re: Spotting base64 encoded JSON, certificates, and private keys
#53I debugged way too many JWT tokens I know eyJhbG by heart
they technically don't need to begin like that! JWT is JSON and is therefore infamously vague... but in practice they for some reason always begin with "alg" so always like eyJhbG
Has anyone tried to send a JWT token with the fields in a different order (e.g. a long key first and key ID and algorithm behind) and see how many implementations will break?
Re: Spotting base64 encoded JSON, certificates, and private keys
#54Mathematically, base64 is such that every block of three characters of raw input will result in four characters of base64'd output.
These blocks can be considered independent of each other. So for example, with the string "Hello world", you can do the following base64 transformations:
* "Hel" -> "SGVs"
* "lo " -> "bG8g"
* "wor" -> "d29y"
* "ld" -> "bGQ="
These encoded blocks can then be concatenated together and you have your final encoded string: "SGVsbG8gd29ybGQ="
(Notice that the last one ends in an equals sign. This is because the input is less than 3 characters, and so in order to produce 4 characters of output, it has to apply padding - part of which is encoded in the third digit as well.)
It's important to note that this is simply a byproduct of the way that base64 works, not actually an intended thing. My understanding is that it's basically like how if you take an ASCII character - which could be considered a base 256 digit - and convert it to hexadecimal (base 16), the resulting hex number will always be two digits long - the same two digits, at that - even if the original was part of a larger string.
In this case, every three base 256 digits will convert to four base 64 digits, in the same way that it would convert to six base 16 digits.
Re: Spotting base64 encoded JSON, certificates, and private keys
#55Reminds me of 1213486160[1]
Besides that, I just spent way too much time figuring out this is an encrypted OpenTofu state. It just looked way too much like a terraform state but not entirely. Tells ya what I spend a lot of time with at work.
This is probably another interesting situation in which you cannot read the state, but you can observe changes and growth by observing the ciphertext. It's probably fine, but remains interesting.
Re: Spotting base64 encoded JSON, certificates, and private keys
#56Good knowledge, now explain why it's like that. {" is ASCII 01111011, 00100010 Base64 takes 3 bytes x 8 bits = 24 bits, groups that 24 bit-sequence into four parts of 6 bits each, and then converts each to a number between 0-63. If there aren't enough bits (we only have 2 bytes = 16 bits, we need 18 bits), pad them with 0. Of course in reality the last 2 bits would be taken from the 3rd character of the JSON string,…
I'd be very hesitant to consider this as some runaway symbol of "CS people being incurious now" over the author simply not being this deeply invested in this at the time of writing in the context of their discovery, especially since it almost certainly doesn't actually matter for them beyond the pattern existing, if even that does.
Re: Spotting base64 encoded JSON, certificates, and private keys
#57Mathematically, base64 is such that every block of three characters of raw input will result in four characters of base64'd output. These blocks can be considered independent of each other. So for example, with the string "Hello world", you can do the following base64 transformations: * "Hel" -> "SGVs" * "lo " -> "bG8g" * "wor" -> "d29y" * "ld" -> "bGQ=" These encoded blocks can then be concatenated together and you…
nitpick but ascii would be base128, largest ascii value is 0x7f which in itself is a telltale if you are looking at hex dumps.
Re: Spotting base64 encoded JSON, certificates, and private keys
#58After staring one time too much at base64-encoded or hex-encoded asn1 I started to believe that scene in the Matrix where operator was looking at raw stream from Matrix at his terminal and was seeing things in it.
Years ago I was part of a group of people I knew who could read and edit large parts of sendmail.cf by hand without using m4. Other people who had to deal with mail servers at the time certainly treated it like a superpower.
In some ways, I miss those days.
Spending hours wrangling sendmail.cf, and finally succeeding, felt like a genuine accomplishment.
Nowadays, things just work, mostly. How boring.
Re: Spotting base64 encoded JSON, certificates, and private keys
#59Well duh. It’s a deterministic encoding. Does not matter if it’s base64, hex, or even rot13.
Is this the state of modern understanding of basic primitives?
Re: Spotting base64 encoded JSON, certificates, and private keys
#60Mathematically, base64 is such that every block of three characters of raw input will result in four characters of base64'd output. These blocks can be considered independent of each other. So for example, with the string "Hello world", you can do the following base64 transformations: * "Hel" -> "SGVs" * "lo " -> "bG8g" * "wor" -> "d29y" * "ld" -> "bGQ=" These encoded blocks can then be concatenated together and you…
nitpick but ascii would be base128, largest ascii value is 0x7f which in itself is a telltale if you are looking at hex dumps.
Yeah, I was aware of that, but I figured it was the easiest way to explain it. It's true that "character representation of a byte" is more accurate, but it doesn't roll off the tongue as easily.