Getting all your data out of Google Reader • Hacker News Board

Re: Getting all your data out of Google Reader

#11

Post by DecoPerson » Fri, Jun 28, 2013, 7:13 PM UTC

Thank you for this! Now I can procrastinate on my own reader app for much longer :)

Should we be concerned with errors like this?

    [W 130629 03:11:54 api:254] Requested item id tag:google.com,2005:reader/item/afe90dad8acde78b (-5771066408489326709), but it was not found in the result

I'm getting ~1-2 per "Fetch N/M item bodies" line.

Re: Getting all your data out of Google Reader

#12

Post by ivank » Fri, Jun 28, 2013, 7:16 PM UTC

drivebyacct2 wrote: post #9

I guess archived RSS data for me isn't terribly important since most people seem to hide the rest of their content behind a "More" link to get those precious ad views.

Really? Pretty much all the feeds I've seen are full-text feeds. For the few that aren't, http://fulltextrssfeed.com/ and http://fullrss.net/ are around.

Re: Getting all your data out of Google Reader

#13

Post by drivebyacct2 » Fri, Jun 28, 2013, 7:48 PM UTC

ivank wrote: post #12

drivebyacct2 wrote: post #9

I guess archived RSS data for me isn't terribly important since most people seem to hide the rest of their content behind a "More" link to get those precious ad views.

Really? Pretty much all the feeds I've seen are full-text feeds. For the few that aren't, http://fulltextrssfeed.com/ and http://fullrss.net/ are around.

Wow! Thank you very much!

Re: Getting all your data out of Google Reader

#14

Post by mihaip » Fri, Jun 28, 2013, 7:56 PM UTC

DecoPerson wrote: post #11

Thank you for this! Now I can procrastinate on my own reader app for much longer :) Should we be concerned with errors like this? [W 130629 03:11:54 api:254] Requested item id tag:google.com,2005:reader/item/afe90dad8acde78b (-5771066408489326709), but it was not found in the result I'm getting ~1-2 per "Fetch N/M item bodies" line.

Usually nothing to worry about, see https://github.com/mihaip/readerisdead/commit/19d3159c985b6e...

Re: Getting all your data out of Google Reader

#15

Post by ijk » Fri, Jun 28, 2013, 7:57 PM UTC

kcvv wrote: post #10

nod wrote: post #8

Thanks mihaip! Worked successfully in Windows CMD for me, without using the \bin shell script: cd C:\mihaip-readerisdead set PYTHON_HOME=C:\mihaip-readerisdead C:\path-to-py27 reader_archive\reader_archive.py --output-directory C:\mystuff Locked up at 251K out of 253K items for me, though. Restarting... success! Looks like it might have locked up trying to start the "Fetching comments" section on my first try.

I'm trying this on windows and seem to be missing base.api module. I can't seem to find this module as well - anyone have a clue where i can get this module?

It's in the \base folder. Set the main folder to be the Python root, as in the grandfather post, and it should be able to find it.

Re: Getting all your data out of Google Reader

#16

Post by kcvv » Fri, Jun 28, 2013, 8:18 PM UTC

ijk wrote: post #15

kcvv wrote: post #10

Earlier quoted context omitted.

I'm trying this on windows and seem to be missing base.api module. I can't seem to find this module as well - anyone have a clue where i can get this module?

It's in the \base folder. Set the main folder to be the Python root, as in the grandfather post, and it should be able to find it.

Thanks! For what ever reason, setting the python root did not help, but i just copied the 'base' folder to python lib folder and that seems to have done the trick.

Re: Getting all your data out of Google Reader

#17

Post by case » Fri, Jun 28, 2013, 8:34 PM UTC

nod wrote: post #8

Thanks mihaip! Worked successfully in Windows CMD for me, without using the \bin shell script: cd C:\mihaip-readerisdead set PYTHON_HOME=C:\mihaip-readerisdead C:\path-to-py27 reader_archive\reader_archive.py --output-directory C:\mystuff Locked up at 251K out of 253K items for me, though. Restarting... success! Looks like it might have locked up trying to start the "Fetching comments" section on my first try.

Similar thing happened to me — it locked up when it was almost to the end, so I killed and restarted the process, and it finished successfully.

Thanks, Mihai!

Re: Getting all your data out of Google Reader

#18

Post by daniel_reetz » Fri, Jun 28, 2013, 9:16 PM UTC

kcvv wrote: post #16

ijk wrote: post #15

Earlier quoted context omitted.

It's in the \base folder. Set the main folder to be the Python root, as in the grandfather post, and it should be able to find it.

Thanks! For what ever reason, setting the python root did not help, but i just copied the 'base' folder to python lib folder and that seems to have done the trick.

I also had success (on W7) using this method. Thank you, mihaip, I am truly grateful.

Re: Getting all your data out of Google Reader

#19

Post by ccera » Sat, Jun 29, 2013, 2:24 AM UTC

kcvv wrote: post #10

nod wrote: post #8

Thanks mihaip! Worked successfully in Windows CMD for me, without using the \bin shell script: cd C:\mihaip-readerisdead set PYTHON_HOME=C:\mihaip-readerisdead C:\path-to-py27 reader_archive\reader_archive.py --output-directory C:\mystuff Locked up at 251K out of 253K items for me, though. Restarting... success! Looks like it might have locked up trying to start the "Fetching comments" section on my first try.

I'm trying this on windows and seem to be missing base.api module. I can't seem to find this module as well - anyone have a clue where i can get this module?

In addition to the above methods, you can copy the base folder and paste it into the same folder as reader_archive.py -- that's what I did and it worked fine.

Re: Getting all your data out of Google Reader

#20

Post by ccera » Sat, Jun 29, 2013, 2:31 AM UTC

Warning to other impatient users:

I didn't read the instructions too well, so the half hour I spent carefully deleting gigantic/uninteresting feeds out of my subscriptions.xml file was all for naught. Because I didn't know I needed to specify the opml_file on the command line, the script just logged into my Reader account (i.e., it walked me through the browser-based authorization process) and downloaded my subscriptions from there -- including all the gigantic/uninteresting subscriptions that I did NOT care to download.

So now I've gone and downloaded 2,592,159 items, consuming 13 GB of space.

I'm NOT complaining -- I actually think it's AWESOME that this is possible -- but if you don't want to download millions of items, be sure to read the instructions and use the opml_file directive.