Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spotting base64 encoded JSON, certificates, and private keys

ergaster.org

Post reply on HN
Print view View original

111–120 of 135 posts

Re: Spotting base64 encoded JSON, certificates, and private keys

#111

Post by userbinator »

cfontes wrote: post #79

Not directly correlated but I know a old guy that can decrypt EBCDIC and credit card positional data format on the fly. And sometimes it was a "feeling" he couldn't explain it properlly but knew exactly the value, name and other data. It was amazing to see him decode VISA and MASTER transactions on the fly in logs and other places.

I can do the same with several proprietary network protocols and data formats I've worked on, as well as some x86 Asm - once you start seeing enough of it, you begin to absorb it almost like learning a language.

Re: Spotting base64 encoded JSON, certificates, and private keys

#112

Post by userbinator »

dhosek wrote: post #103
anitil wrote: post #100

Earlier quoted context omitted.

> the character you'd get on a traditional U.S. keyboard layout I use a different layout so I'd never realised there was method to the madness! I get the following $ echo -n ' !@#$%^&*(' | xxd -p 2021402324255e262a28

It’s more the old TTY layout which differs somewhat from the modified typewriter layout that’s become standard for computer keyboards. The old Apple ][ keyboard had 1–9 corresponding to the next row in ASCII, shift-0 was @, I think other characters were ±16 based on shift. Early ASCII implementations were often slightly inconsistent but codings were often based on keyboard layouts.

The order of the punctuation descends from the very first typewriters, in the late 19th century:

https://en.wikipedia.org/wiki/File:Remington_2_typewriter_ke...

Re: Spotting base64 encoded JSON, certificates, and private keys

#113

Post by mdaniel »

Faaak wrote: post #36

Nitpick, but enclosing the first string in single quotes would make the reading better: $ echo '{"' | base64 Vs $ echo "{\"" | base64

That's a huge pet peeve of mine, but similar to the control-r comment elsewhere, I have just come to terms with the fact that most developers are allergic to shell

Re: Spotting base64 encoded JSON, certificates, and private keys

#114

Post by ruszki »

syncsynchalt wrote: post #71
netsharc wrote: post #45

Good knowledge, now explain why it's like that. {" is ASCII 01111011, 00100010 Base64 takes 3 bytes x 8 bits = 24 bits, groups that 24 bit-sequence into four parts of 6 bits each, and then converts each to a number between 0-63. If there aren't enough bits (we only have 2 bytes = 16 bits, we need 18 bits), pad them with 0. Of course in reality the last 2 bits would be taken from the 3rd character of the JSON string,…

I think the audience already understands why it works, it's more the knowing there's a relatively small set of mnemonics for these things that's interesting. "eyJ" for JSON, "LS0" for dashes (PEM encoding), "MII" for the DER payload inside a PEM, and so on. I've been doing this a long time but until today the only one I'd noticed was "MII".

The audience yes, but the author clearly seems to not understand it when they wrote this the first time.

> I did a few tests in my terminal, and he was right!

He clearly had no clue how base64 worked. You don’t need a test, if you know it.

> As pointed out by gnabgib and athorax on Hacker News, this actually detects the leading dashes of the PEM format

They needed help for this. I’m not sure that they opened Wikipedia at last to understand how base64 works even now. The whole article has an “it’s magic!” vibe.

Re: Spotting base64 encoded JSON, certificates, and private keys

#115

Post by rini17 »

cogman10 wrote: post #63
xg15 wrote: post #52

Earlier quoted context omitted.

I think it's more the waste of space in it all. Encoding data in base64 increases the length by 33%. So base64-encoding twice will blow it up by 33% of the original data and then again 33% of the encoded data, making 69% in total. And that's before adding JSON to the mix... And before "space is cheap": JWT is used in contexts where space is generally not cheap, such as in HTTP headers.

Precisely my thoughts. You have to ask the question "why are we encoding this as base64 in the first place?" The answer to that is generally that base64 plays nice with http headers. It has no newlines or special characters that need special handling. Then you ask "why encode json" And the answer is "because JSON is easy to handle". Then you ask the question "why embed a base64 field in the json?" And the answer is "…

JSON doesn't even handle text...

Re: Spotting base64 encoded JSON, certificates, and private keys

#116

Post by themk »

dhosek wrote: post #37

Kind of reminds me of a junior being amazed when I was able to read ascii strings out of a hex stream. Us old folks have seen a lot .

I used to be able to read ascii flying over a uart using an oscilloscope. I think these days the scopes will decode it for you.

Good times.

Re: Spotting base64 encoded JSON, certificates, and private keys

#117

Post by karel-3d »

xg15 wrote: post #53
karel-3d wrote: post #51

Earlier quoted context omitted.

they technically don't need to begin like that! JWT is JSON and is therefore infamously vague... but in practice they for some reason always begin with "alg" so always like eyJhbG

Has anyone tried to send a JWT token with the fields in a different order (e.g. a long key first and key ID and algorithm behind) and see how many implementations will break?

there are better things to do, like send json that has "alg" twice, each different (one of them "none" ideally) and different implementations handle it differently

Re: Spotting base64 encoded JSON, certificates, and private keys

#118

Post by mschuster91 »

gnabgib wrote: post #14

You can spot Base64 encoded JSON. The PEM format (that begins with `-----BEGIN [CERTIFICATE|CERTIFICATE REQUEST|PRIVATE KEY|X509 CRL|PUBLIC KEY]-----`) is already Base64 within the body.. the header and footer are ASCII, and shouldn't be encoded[0] (there's no link to the claim so perhaps there's another format similar to PEM?) You can't spot private keys, unless they start with a repeating text sequence (or use the…

> The PEM format (that begins with `-----BEGIN [CERTIFICATE|CERTIFICATE REQUEST|PRIVATE KEY|X509 CRL|PUBLIC KEY]-----`) is already Base64 within the body.. the header and footer are ASCII, and shouldn't be encoded[0] (there's no link to the claim so perhaps there's another format similar to PEM?)

In practice, you will spot fully b64 encoded PEMs all the time once you have Kubernetes in play... create a Secret from a file and that's what you will find.

Re: Spotting base64 encoded JSON, certificates, and private keys

#119

Post by sans_souse »

pedropaulovc wrote: post #84
mmastrac wrote: post #10

I built a JWT support library at work ( https://github.com/geldata/gel-rust/tree/master/gel-jwt ) and I can confirm that JWTs all sound like "eyyyyyy" in my head.

Ey, I'm JSON!

Eyy, JSON I'm PTER, you don't know me yet but just you wait. Eyyyy.

Re: Spotting base64 encoded JSON, certificates, and private keys

#120

Post by DPDmancul »

thibaultamartin wrote: post #31
athorax wrote: post #13

Base64 encoded yaml files will also be LS-prefixed if they have the document marker (---)

That's right, I've added an errata to clarify. Thanks for the heads up!

You can increase the guess accuracy a little by looking for the "tLS" characters , skipping the first 3 chars. Also this is a mnemonic about TLS and identifies all strings starting with 5 dashes, excluding so most of yaml documents
Post reply on HN