Live data from Hacker News

Hacker News Official API

github.com

121–130 of 138 posts

Re: Hacker News Official API

#121
post #70

In response to people's complaints about the usability of the HN Firebase API: yes. We're going to eventually have a new API that returns a simple JSON version of any HN URL. At that point we'll phase out the Firebase API, with a generous deprecation period. I'd be curious to hear people's thoughts about what a generous deprecation period might be.

How difficult would it be to host the same Firebase API on top of whatever will be driving the new JSON stuff?

If you mean the documented API, it shouldn't be hard. If you mean reimplementing Firebase, that would be another matter.

Re: Hacker News Official API

#122
post #70

In response to people's complaints about the usability of the HN Firebase API: yes. We're going to eventually have a new API that returns a simple JSON version of any HN URL. At that point we'll phase out the Firebase API, with a generous deprecation period. I'd be curious to hear people's thoughts about what a generous deprecation period might be.

Can you allow login and write operation via the API? I want to build a HN web client but it's not possible without scrapping the site which won't scale.

Probably not in the first release, but hopefully after that.

Re: Hacker News Official API

#123
post #120

Earlier quoted context omitted.

The great thing about the Firebase API is that it's live, so you can subscribe to updates to a post/comment/profile and see them as they're made (no polling, just websockets). I'd hope any future version would retain this feature. I'm reluctant to lose it as I'm about to release a browser extension that relies on it pretty heavily :) [Edit]: this feature is undocumented in the link above, but it's a standard feature…

Hmm. I'm sorry but I can't promise that. It isn't part of the HN API, though I can understand why people use it since Firebase offers it.

So is that to say it's just an unintended consequence of the data being hosted on Firebase? That's wild. Especially as I'd be surprised if most integrations aren't using this feature (IMO it's by and far the best feature of the API).

To demonstrate, here's an example in 18 lines of code that v. efficiently listens to live updates to a profile (yours): https://codepen.io/theprojectsomething/pen/xxWBOWN?editors=1...

Re: Hacker News Official API

#124
post #70

In response to people's complaints about the usability of the HN Firebase API: yes. We're going to eventually have a new API that returns a simple JSON version of any HN URL. At that point we'll phase out the Firebase API, with a generous deprecation period. I'd be curious to hear people's thoughts about what a generous deprecation period might be.

6 to 12 months AFTER we have dark mode.

Re: Hacker News Official API

#125

Wish API would show comment votes instead of only showing them up until 8 years ago when displaying them was turned off on the main site. Aside from allowing users to enable them via their on interface, do analysis, etc — it broke HN’s comment search based on comment popularity. Related: https://news.ycombinator.com/item?id=32535637

I’d like to be able to conveniently access my most popular comments. There are quite a few things in there that I might like to write longer-form articles about. ( https://james.darpinian.com/blog/scraping-my-own-hacker-news... gives one way of doing it, by crawling your /threads?id=‹username› page, which does contain the point counts for each comment by you.)

(Aside: I finally got round to scraping my comments in that way. My comments follow the sort of distribution you’d expect, ranging from −4 to +76, but there’s one outlier at +283, https://news.ycombinator.com/item?id=27274146. People really hate stale bots on bug trackers.)

Re: Hacker News Official API

#126

Only using the official API is quite cumbersome, as fetching a story with comments requires one network request per each comment. Instead, try considering the (more unknown) Algolia API[0]. It supports fetching the entire tree in one request. Unfortunately the comments aren’t sorted, so you’ll still have to use the official one (or parse the HN HTML) if you need that. For my HN client[1], I parse the HN HTML (as it’s…

If you already parse the html, what's the point in calling the Algolia API at all?

I believe HN's website has a rate-limiter even for viewing, which the HN API doesn't have.

Re: Hacker News Official API

#127
post #94
post #91

Earlier quoted context omitted.

Ugh no... No idea how I've missed that. Thanks!

I was actually wondering how I knew about it :p I'm not sure it's linked anywhere visible, but it does have an entry in the html head....

There's also /showrss

I'm not sure if there are more tho, I know /ask doesn't have one.

Re: Hacker News Official API

#128
post #120

Earlier quoted context omitted.

Hmm. I'm sorry but I can't promise that. It isn't part of the HN API, though I can understand why people use it since Firebase offers it.

So is that to say it's just an unintended consequence of the data being hosted on Firebase? That's wild. Especially as I'd be surprised if most integrations aren't using this feature (IMO it's by and far the best feature of the API). To demonstrate, here's an example in 18 lines of code that v. efficiently listens to live updates to a profile (yours): https://codepen.io/theprojectsomething/pen/xxWBOWN?editors=1...

It was certainly an unintended consequence of the data being hosted on Firebase. I'm not saying it isn't a valuable feature! And if people are doing interesting and useful things with it, I'm open to implementing something to keep that going—as long as it's simple and there's a clear line between it and "reimplement Firebase", which would be a foolish thing for us to try to undertake.

Re: Hacker News Official API

#130

I wrote a simple hn data downloaded based on this api. Last I remember I was able to download the entire corpus by letting it run overnight. https://github.com/ashish01/hn-data-dumps

Oh this might be exactly what I need. Giving it a try now. Thanks!
Post reply on HN