Hacker News API • Hacker News Board

Re: Hacker News API

#31

Post by pbnjay » Tue, Oct 07, 2014, 6:16 PM UTC

Great news! Can we add endpoints for "Show HN" and "Ask HN" too though? I don't see an easy way to get them.

Re: Hacker News API

#32

Post by 650REDHAIR » Tue, Oct 07, 2014, 6:16 PM UTC

kevin wrote: post #27

bambax wrote: post #10

Any kind of rate limitations we should be made aware of?

Rate limit is up to Firebase. We'll see if we can get them to answer.

That doesn't worry you at all? Were any alternatives considered?

Re: Hacker News API

#33

Post by redox_ » Tue, Oct 07, 2014, 6:17 PM UTC

jaredsohn wrote: post #9

zuck9 wrote: post #7

Earlier quoted context omitted.

Algolia scrapes HN HTML and provides their own API from what it looks like.

They don't scrape it. From the hnsearch about page: "Every 2 minutes, the last 1000 HN items (stories, comments, polls) are sent to Algolia's indexing API. Items from the last 48 hours are refreshed every hour."

Correct, and we're planning to move to the new official API ASAP (instead of the legacy one -> it was not web crawler but far from perfect) for the indexing.

Regarding the REST APIs, let's keep both for now :)

Re: Hacker News API

#34

Post by minimaxir » Tue, Oct 07, 2014, 6:19 PM UTC

christiangenco wrote: post #30

Oh man you guys, patio11 has generated massive amounts of content: https://hacker-news.firebaseio.com/v0/user/patio11.json?prin... I count 8,483 submissions. I'm sure there's something interesting to be done with all of this data. A word frequency chart? --- Edit: So apparently there's a ruby gem that lets you feed it a body of text and generates pseudo-random phrases based on that text. I present to you the patio11…

In February, I did an analysis on all Hacker News stories: http://minimaxir.com/2014/02/hacking-hacker-news/

Yesterday, I published an analysis on all Hacker News comments: http://minimaxir.com/2014/10/hn-comments-about-comments/

There's a lot of interesting trends in the data. Let me know if you want to know anything in particular and I'll get back to you. :)

Re: Hacker News API

#35

Post by airlocksoftware » Tue, Oct 07, 2014, 6:20 PM UTC

This... is cool, but also kinda sucks for me. I've invested dozens of hours into writing an extremely complicated scraper for my Android version of HN.

https://play.google.com/store/apps/details?id=com.airlocksof...

The newest version (still under development, probably a month or two from release) adds support for displaying polls, linking to subthreads, and full write support (voting, commenting, submitting, etc). I'm fine with switching to a new API (Square's Retrofit will make it super easy to switch), but without submitting, commenting, and upvote support I have to disable a bunch of features I worked really hard on. Also it would've been cool to know this was coming about 3 months ago so I didn't waste my time.

Anyways, quick question on how it works -- when I query for the list of top stories

https://hacker-news.firebaseio.com/v0/topstories.json?print=...

it just returns a list of ids. Do I have to make a separate request for each story

https://hacker-news.firebaseio.com/v0/item/8863.json?print=p...)

to assemble them into a list for the front page, or am I missing something?

Re: Hacker News API

#36

Post by kogir » Tue, Oct 07, 2014, 6:21 PM UTC

650REDHAIR wrote: post #32

kevin wrote: post #27

Earlier quoted context omitted.

Rate limit is up to Firebase. We'll see if we can get them to answer.

That doesn't worry you at all? Were any alternatives considered?

Trust me, their rate limit will be much higher than anything we'd easily be able to provide.

Re: Hacker News API

#37

Post by tlo » Tue, Oct 07, 2014, 6:21 PM UTC

For my twitter bot https://twitter.com/hn_bot_top1 I use http://api.ihackernews.com/ at the moment. This works but the site (and the API) is quite often not available. So I'll probably switch to the new API as soon as I have some spare time...

Re: Hacker News API

#38

Post by dabit3 » Tue, Oct 07, 2014, 6:21 PM UTC

I love firebase!

Re: Hacker News API

#39

Post by dimillian » Tue, Oct 07, 2014, 6:22 PM UTC

So, does it mean I can get the top stories, and then get a top story item with all the comment expanded ? I mean, at first it look like it just send me the id and I need to fetch the detail for each of them. Again, this is just looking at the REST API, not the iOS SDK for example.

I'll need to "convert" SwiftHN (https://github.com/Dimillian/SwiftHN) either to this new API or adapt my scrapping engine to the new site layout.

Re: Hacker News API

#40

Post by sararob » Tue, Oct 07, 2014, 6:22 PM UTC

bambax wrote: post #10

Any kind of rate limitations we should be made aware of?

[Firebase Dev Advocate here] We don't currently have rate limits. I'd recommend using our SDKs, they handle connections more efficiently than dealing with the REST API. You can also run your own server process using our Node (https://www.npmjs.org/package/firebase) or Java libraries (https://www.firebase.com/docs/android/quickstart.html).