Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse Engineer Amazon's Whispersync

ptbrowne.github.io

Post reply on HN
Print view View original

11–20 of 38 posts

Re: Reverse Engineer Amazon's Whispersync

#11

Post by ptbrowne »

kilroy123 wrote: post #3

I'm very interested in this. I read a lot of books that aren't from Amazon on my Kindle. I do a lot of highlighting, which I like to export to readwise. In order to do this I have to plug in my Kindle. Mount it. Find the clippings file. Upload the file to readwise. The best I came up with was a script that detects when I plug-in my Kindle and opens the finder window and opens the readwise page. Pretty lame.

Unfoturnately, Amazon does not sync highlights on books that have not been purchased on Amazon. This is also why Readwise cannot automatically sync those highlights.

https://help.readwise.io/article/67-some-of-my-kindle-books-...

Re: Reverse Engineer Amazon's Whispersync

#12

Post by ptbrowne »

rektide wrote: post #9

would this work if Amazon was using cert pinning? cert pinning basically seems like a way to make apps able to resist user's having any freedom or power. the author here installs a custom CA to man-in-the-middle. now more & more apps use cert pinning, which would block that, I believe, & keep the user from being able to reverse engineer & probe. imo the forces in favor of securing things have done the users great gre…

You're right, this technique would not work if Amazon had used cert pinning. I haven't yet tried to bypass a cert pinning app, but it seems that frameworks like Frida could help : https://medium.com/@ved_wayal/hail-frida-the-universal-ssl-p....

Re: Reverse Engineer Amazon's Whispersync

#13

Post by bredren »

ptbrowne wrote: post #10
tshrjn wrote: post #6

readwise kinda helps do something similar?

Hi, thanks I did not know Readwise. I signed up for an account to check how it works. It uses a browser extension to sync the highlights instead of using the API. It can sync automatically but your browser must stay open (which is OK IMHO). Readwise can sync much more sources also. Thanks for introducing me to Readwise, I might use it :)

Still a valuable exploration!

That said, this suggestion and it’s method of avoiding direct interaction, but instead using rendered content in browser is another reminder of how powerful browser extensions are.

They can also make additional requests using the user’s ip address.

Would pulling data this way also seems less likely to be a TOS issue?

Reason being that you are reading content you are manipulating or displaying in a browser “after” amazon does their own highlighting.

Re: Reverse Engineer Amazon's Whispersync

#14

Post by Godel_unicode »

ptbrowne wrote: post #12
rektide wrote: post #9

would this work if Amazon was using cert pinning? cert pinning basically seems like a way to make apps able to resist user's having any freedom or power. the author here installs a custom CA to man-in-the-middle. now more & more apps use cert pinning, which would block that, I believe, & keep the user from being able to reverse engineer & probe. imo the forces in favor of securing things have done the users great gre…

You're right, this technique would not work if Amazon had used cert pinning. I haven't yet tried to bypass a cert pinning app, but it seems that frameworks like Frida could help : https://medium.com/@ved_wayal/hail-frida-the-universal-ssl-p... .

Or just open up the apk in e.g. apk studio and replace the bundled cert with the one from your mitm proxy. Effectively, that'll repin the app to your proxy specifically. There are more exotic pinning methods that this won't solve (the forever cat-and-mouse game) but it's good for many apps.

Re: Reverse Engineer Amazon's Whispersync

#16

Post by farss »

sushicalculus wrote: post #7
kilroy123 wrote: post #3

I'm very interested in this. I read a lot of books that aren't from Amazon on my Kindle. I do a lot of highlighting, which I like to export to readwise. In order to do this I have to plug in my Kindle. Mount it. Find the clippings file. Upload the file to readwise. The best I came up with was a script that detects when I plug-in my Kindle and opens the finder window and opens the readwise page. Pretty lame.

Could you post this script to somewhere? Gist, codepen, etc? I want to see if can improve it. There might be a way to have the terminal auto-copy the clippings file

Here's a similar script I wrote. Would be glad for feedback.

https://github.com/wneuheisel/Kindle-Notes-Backup

Re: Reverse Engineer Amazon's Whispersync

#17

Post by kilroy123 »

farss wrote: post #16
sushicalculus wrote: post #7

Earlier quoted context omitted.

Could you post this script to somewhere? Gist, codepen, etc? I want to see if can improve it. There might be a way to have the terminal auto-copy the clippings file

Here's a similar script I wrote. Would be glad for feedback. https://github.com/wneuheisel/Kindle-Notes-Backup

Oh awesome. I see you spent much longer than my 30 minutes on this. I'm going to take a good look at this later.

Re: Reverse Engineer Amazon's Whispersync

#18

Post by syoc »

fprog wrote: post #4

Loved section 1.5.1, "Bytes in color". The author wrote code to help them understand a custom binary format by coloring the bytes based on their value, making patterns in the format visually obvious. The structure of the binary becomes much more clear when visualized this way. That's a handy approach I've mentally bookmarked for the future.

You might be interested in https://github.com/sharkdp/hexyl

> hexyl is a simple hex viewer for the terminal. It uses a colored output to distinguish different categories of bytes (NULL bytes, printable ASCII characters, ASCII whitespace characters, other ASCII characters and non-ASCII).

Re: Reverse Engineer Amazon's Whispersync

#20

Post by yellow_lead »

ptbrowne wrote: post #12
rektide wrote: post #9

would this work if Amazon was using cert pinning? cert pinning basically seems like a way to make apps able to resist user's having any freedom or power. the author here installs a custom CA to man-in-the-middle. now more & more apps use cert pinning, which would block that, I believe, & keep the user from being able to reverse engineer & probe. imo the forces in favor of securing things have done the users great gre…

You're right, this technique would not work if Amazon had used cert pinning. I haven't yet tried to bypass a cert pinning app, but it seems that frameworks like Frida could help : https://medium.com/@ved_wayal/hail-frida-the-universal-ssl-p... .

This module works well too. https://github.com/Fuzion24/JustTrustMe
Post reply on HN