Live data from Hacker News

Hacker News API

blog.ycombinator.com

61–70 of 315 posts

Re: Hacker News API

#62
If up/down vote data were included in the API, much needed experimentation on collaborative filtering would be made possible! This is Hacker News after all.

Right now one team, Ycombinator, is trying to fix important issues in the ranking and moderation of posts and comments. Many of us are frustrated by the increasing domination of popularity (and hatred) over quality and relevance. A lot of good submissions and comments are simply buried, never to be found. There is too much muck to have to wade through. The timing of posts and comments plays a much larger role than quality. I could go on and on.

Imagine a Netflix Prize-like flowering of experiments and collaboration, leveraging the hacker community's collective smarts and enthusiasm. Many of us have ideas, but right now are unable to test them. What a shame if a great idea dies on a notepad.

There are two possible issues with opening up voting data: gaming and privacy. If having vote data allows someone to game the front page, then only include it with some delay (2 days?) so that it could't be used to game the front page. This will still allow experimentation with collaborative filtering algorithms and the like.

My take on the privacy issue is that anonymity isn’t that important for a site like Hacker News:

1. Startup culture is about straight talk, putting your money where your mouth is, and open critical feedback, both in the giving and receiving. There are precedents for exposing voting data (e.g. Quora, Facebook, Stack Exchange).

2. HN is not aimed at political discussions or other topics where anonymity can be paramount.

3. Pseudonymity is sufficient for those who don’t want their votes and comments tied back to their actual identity.

Thoughts?

I would love to hear from others who yearn to experiment with alternate algorithms and strategies for improving Hacker News.

Re: Hacker News API

#63
Does anyone have a good solution on how to get around making a separate request for each Item? Is there somewhere we can pass an array of Item Ids? Is this planned for the future?

Thanks HN!

EDIT: I just read the post about using the Firebase SDK to do this efficiently.

Re: Hacker News API

#64

Oh man you guys, patio11 has generated massive amounts of content: https://hacker-news.firebaseio.com/v0/user/patio11.json?prin... I count 8,483 submissions. I'm sure there's something interesting to be done with all of this data. A word frequency chart? --- Edit: So apparently there's a ruby gem that lets you feed it a body of text and generates pseudo-random phrases based on that text. I present to you the patio11…

tptacek is similarly impressive, with 30,051 posts. https://hacker-news.firebaseio.com/v0/user/tptacek.json?prin...

"Impressive" is one word for it, I guess.

Re: Hacker News API

#65
post #50

I've been working on a Hacker News client for Windows Phone over the past several weeks and am very close to an initial release, so I feel somewhat ambivalent about this. On the one hand, of course it's great that HN is finally getting a proper API and also modernizing its markup (which is a mess even if you ignore all the tables – for example, the first paragraph in a comment usually isn't wrapped in tags), but on t…

Use the Firebase libraries rather than the REST one to efficiently handle requests. I believe it uses a websocket internally. "It does all the work for you and is awesome." to quote Nick.

Re: Hacker News API

#66
To everyone asking about logged-in access and write access: this is just a first release! Where it goes from here will depend, in good iterative fashion, on what people want.

Re: Hacker News API

#68

How does this differ from the Algolia HN API in terms of data access? ( https://hn.algolia.com/api ) I was able to download all HN data recently with ease using that endpoint. Authentication? EDIT: After looking at the documentation there are two new aspects of the Firebase API not in the Algolia API: 1) Ability to see deleted/dead stories. 2) Endpoint for user data. Question to kogir/dang: Has the "delay" field (Del…

The 'delay' field has been there for years. I vaguely remember pg announcing it here when he added it, so we could probably dig up the exact date.

Re: Hacker News API

#69

I really appreciate giving a 3 week heads up before moving to a new frontend structure. It's a nice gesture, but I have this horrible feeling that there's only about a 10% chance that my Hacker News app gets updated in time. I know you can't not iterate because people are scraping, but it does stink. At least this will make everything more future-proof going forward. However, it may be nice to give a bit more heads u…

As someone who poured countless hours into meticulously scraping the HN markup and faces the prospect of having to port all my code with dread, I'll probably be pleading for an extension alongside you.

Re: Hacker News API

#70
post #65
post #50

I've been working on a Hacker News client for Windows Phone over the past several weeks and am very close to an initial release, so I feel somewhat ambivalent about this. On the one hand, of course it's great that HN is finally getting a proper API and also modernizing its markup (which is a mess even if you ignore all the tables – for example, the first paragraph in a comment usually isn't wrapped in tags), but on t…

Use the Firebase libraries rather than the REST one to efficiently handle requests. I believe it uses a websocket internally. "It does all the work for you and is awesome." to quote Nick.

Also, while they don't have a Windows Phone API I previously wrote a COM Windows Scripting API to use it via Chackra from an ASP.Net app.

I think it's even easier now that JS is a supported app language.

Post reply on HN