Hacking Hacker News

Post by sillysaurus3 » Sun, Feb 23, 2014, 3:41 AM UTC

minimaxir wrote: post #19

sillysaurus3 wrote: post #11

Earlier quoted context omitted.

When trying to access page 2 via that endpoint: http://hn.algolia.com/api/v1/search_by_date?tags=story&hitsP... "you can only fetch the 1000 hits for this query, contact us to increase the limit" It was a nice try, but it did seem too good to be true.

You can paginate using the created_at_i parameter (edited OP) Just pass created_at_i I was able to download 500k stories (i.e. about half of HN's 1.26M stories) before I ran into memory issues; I've fixed them and am downloading the rest.

Post by sosborn » Sun, Feb 23, 2014, 4:15 AM UTC

ThomPete wrote: post #20

rickdale wrote: post #14

This is totally cool and kudos to you. But be aware there are limits of personalized hacker news. I think Bill Maher put it best when ranting just last night about facebooks customized news feeds: Newspapers may be old-fashioned, but here's what we're losing if you never see one. They are trying to tell you what's actually important, not just what's important to you. You may not read the whole paper, but you at least…

Newspapers are trying to show you what they think is actually important. But of course omnibus papers will never really tell you what is important only what is current.

Post by hnriot » Sun, Feb 23, 2014, 4:46 AM UTC

rickdale wrote: post #14

This is totally cool and kudos to you. But be aware there are limits of personalized hacker news. I think Bill Maher put it best when ranting just last night about facebooks customized news feeds: Newspapers may be old-fashioned, but here's what we're losing if you never see one. They are trying to tell you what's actually important, not just what's important to you. You may not read the whole paper, but you at least…

Post by ehsanu1 » Sun, Feb 23, 2014, 5:32 AM UTC

minimaxir wrote: post #2

FYI, the new Hacker News API allows easy programmatic access of story/comments and infinite chronological paging. You could download every Hacker News story in less than 2 hours without breaking the API request limit. https://hn.algolia.com/api

    RATE LIMITS
    We are limiting the number of API requests from a single IP to 1000 per hour.
    If you or your application has been blacklisted and you think there has been
    an error, please contact us.

Post by seizethecheese » Sun, Feb 23, 2014, 5:58 AM UTC