Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Decrypting Blind's Encrypted API

blog.jldc.me

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Decrypting Blind's Encrypted API

#21

Post by kccqzy »

Pfhreak wrote: post #18
kccqzy wrote: post #16

This is yet another reminder that good JS minification tools exist that can absolutely change object properties into short minimal strings instead of descriptive names. It's called the Closure Compiler in advanced mode. You do have to have quite a bit of discipline in writing the JS to have that though. Some languages like ClojureScript actually do this by default, so it doesn't take much effort. Also it helps if you…

The thought of the latter just made me shudder. Removing keys locks your API in really unpleasant ways. More importantly, it's less human readable and harder to reason about. Please don't do this unless you've got a very specific need for the performance.

Keys are not really removed in code, only from the wire format. In code you can still write `myObject.property` but the minifier translates that to say, `a[6]` instead. Naturally there would be tools for the developer to translate these arrays back into full objects in the case of debugging in production. Harder to reason about yes, but few devs would reason about their code using minified code, why should they reason about using minified wire format?

(Of course I must admit that this is only suitable for private APIs, not APIs published to explicitly allow third parties to use.)

Have you tried looking at, say, Gmail's XHR requests and responses?

Re: Decrypting Blind's Encrypted API

#22

Post by techslave »

bowmessage wrote: post #20
SlowRobotAhead wrote: post #13

Earlier quoted context omitted.

This kind of deep packet inspection is in no way limited to phones.

sure, my point was really: there's no way for your employer to intercept the packet contents unless they've got their own cert bundle installed, and you'd have to be pretty clueless to use this from a corp machine of any kind.

most people are in fact clueless.

Re: Decrypting Blind's Encrypted API

#23

Post by pheug »

jiofih wrote: post #2

Is there any point in encrypting API payloads when the traffic is going via TLS?

Security through obscurity. Pretty sure Blind did this just to piss off scrapers. I mean I've been scraping them for some time until they pulled this encryption trick about a year ago. I didn't have time to reverse engineer their js and keep my scraper up to date, that'd take way more time than it took me to write some 100 lines to scrape them originally. So score 1 for them.

Re: Decrypting Blind's Encrypted API

#24

Post by fastball »

kccqzy wrote: post #21
Pfhreak wrote: post #18

Earlier quoted context omitted.

The thought of the latter just made me shudder. Removing keys locks your API in really unpleasant ways. More importantly, it's less human readable and harder to reason about. Please don't do this unless you've got a very specific need for the performance.

Keys are not really removed in code, only from the wire format. In code you can still write `myObject.property` but the minifier translates that to say, `a[6]` instead. Naturally there would be tools for the developer to translate these arrays back into full objects in the case of debugging in production. Harder to reason about yes, but few devs would reason about their code using minified code, why should they reaso…

If you're going to do that, why not just forego human readability entirely and user an interchange format like protobuf or flatbuffers?

Re: Decrypting Blind's Encrypted API

#25

Post by kccqzy »

fastball wrote: post #24
kccqzy wrote: post #21

Earlier quoted context omitted.

Keys are not really removed in code, only from the wire format. In code you can still write `myObject.property` but the minifier translates that to say, `a[6]` instead. Naturally there would be tools for the developer to translate these arrays back into full objects in the case of debugging in production. Harder to reason about yes, but few devs would reason about their code using minified code, why should they reaso…

If you're going to do that, why not just forego human readability entirely and user an interchange format like protobuf or flatbuffers?

Because if you use the protobuf wire format, all the parsing code needs to be written in JS and won't be as performant as JSON parsing built into the browser.

But yes you can in fact transform protocol buffers into JS arrays in the way I described. I'm essentially describing protobuf designed for JS. Imagine your protobuf definitions are read by a compiler which spits out JS classes with getters and setters. These getters and setters access the underlying array with an assigned index. Your minifier inlines these getters and setters into direct array access. Voilà.

Re: Decrypting Blind's Encrypted API

#27

Post by artificial »

userbinator wrote: post #15
eralps wrote: post #7

Nice article! I always wonder what the legal aspects of publishing a reverse engineering article for a private API are? Does the company that the API belongs to have rights to an obligatory take down request?

Is it really "private" if everyone with a browser and a brain can see what it's doing...?

This is what's frustrating about accessing content online. Is it fair game if it's on a web server since the requester cannot determine intent? Legally it doesn't appear so.

Re: Decrypting Blind's Encrypted API

#28

Post by yonder »

kccqzy wrote: post #25
fastball wrote: post #24

Earlier quoted context omitted.

If you're going to do that, why not just forego human readability entirely and user an interchange format like protobuf or flatbuffers?

Because if you use the protobuf wire format, all the parsing code needs to be written in JS and won't be as performant as JSON parsing built into the browser. But yes you can in fact transform protocol buffers into JS arrays in the way I described. I'm essentially describing protobuf designed for JS. Imagine your protobuf definitions are read by a compiler which spits out JS classes with getters and setters. These ge…

Why do you assume that native JSON parsing is faster than protobuf parsing using JS?

In most cases protobuf is faster.

Re: Decrypting Blind's Encrypted API

#29

Post by seangrogg »

yonder wrote: post #28
kccqzy wrote: post #25

Earlier quoted context omitted.

Because if you use the protobuf wire format, all the parsing code needs to be written in JS and won't be as performant as JSON parsing built into the browser. But yes you can in fact transform protocol buffers into JS arrays in the way I described. I'm essentially describing protobuf designed for JS. Imagine your protobuf definitions are read by a compiler which spits out JS classes with getters and setters. These ge…

Why do you assume that native JSON parsing is faster than protobuf parsing using JS? In most cases protobuf is faster.

Having spent a LOT of time looking at this, browsers come with built-in (nil cost) JSON parsers.

You need a proto parsing lib and a collection of .proto schemas to even begin using protobufs, so you need to be dealing with at least that much data saved before proto even starts being a win. While the parsing lib can be cached and is largely a rounding error over a long term, every iteration to the .proto files means fetching a new version which contains all the contents of the previous version (or else sacrificing backwards compatibility).

Beyond the additional payload costs you also have to factor in the API itself. Any win to keys can largely be obtained via compression so that's only a nominal win. APIs with many string values are not going to see many benefits, either, and may actually be better served by compression. The real win for proto is in large numbers but there aren't many APIs using many values in the 256-65k range (let alone higher). Proto does do really well with booleans and null, though. Unpacked arrays aren't a really strong win for them, either (though packed ones are a win for large arrays). They also have weird quirks for maps that don't let them achieve parity with JSON, IIRC.

Parsing time is not a huge win given normal API response sizes. I was parsing a JSON blob with 100k values four years ago on a shitty Dell in 2 seconds and can't think of anything near that size in the wild. Most API responses are going to be parsed faster than human perception rendering the point mostly moot.

The real win is the direct impact to spend on bandwidth that scales with size, but that comes at the cost of developer productivity and not everyone has Google's warchest and can afford SWEs memeing about how they get promoted by spending 2 years updating protos.

Having worked at Google, Protobuf is a solid choice when you're working in multiple languages on multiple internal machines and haven't already bought into other means of serializing data. But they do not particularly shine when targeting browsers unless there is a LOT of data going back and forth and your front-end engineering team doesn't mind working around jspb's quirks, opaque errors, and subtle nuances.

Re: Decrypting Blind's Encrypted API

#30

Post by seangrogg »

tehlike wrote: post #14
sonicggg wrote: post #12

Earlier quoted context omitted.

You'd think that engineers from top - tier tech companies would know better, before sharing sensitive information on some random website.

People like venting.

Others don't even care if the info was de-anonymized in the first place and just enjoyed the topics that were more openly discussed there.
Post reply on HN