Live data from Hacker News

Hacker News Official API

github.com

101–110 of 138 posts

Re: Hacker News Official API

#101
post #97

What is the use case? I can't imagine anything besides getting a real time view on the frontpage or likes of my own comments that a monthly static data dump wouldn't solve.

Probably 3rd party clients that offer quality of life / usability improvements over the website. E.g. I use Hackers [1][2] because it is much easier to read stories on mobile and supports dark mode. Unfortunately, it doesn't support commenting or downvoting (yet?) [1] https://apps.apple.com/us/app/hackers-for-hacker-news/id6035... [2] https://github.com/weiran/Hackers

[deleted]

Re: Hacker News Official API

#102
post #70

In response to people's complaints about the usability of the HN Firebase API: yes. We're going to eventually have a new API that returns a simple JSON version of any HN URL. At that point we'll phase out the Firebase API, with a generous deprecation period. I'd be curious to hear people's thoughts about what a generous deprecation period might be.

I'd say it depends on how straightforward the new API will be. The app I use (Materialistic) is both free and ads-free. So I don't expect the developer to drop everything they are doing to update it if will require a lot of work. So 6 months minimum if everything is well documented, a year if it will be a hassle.

I use materialistic also, but as far as I'm aware that hasn't been an update to the app in years. In addition I believe the materialistic app listing on Google Play is also been removed.

Do you have some updated version that I'm unaware of?

Re: Hacker News Official API

#103
post #97

What is the use case? I can't imagine anything besides getting a real time view on the frontpage or likes of my own comments that a monthly static data dump wouldn't solve.

Probably 3rd party clients that offer quality of life / usability improvements over the website. E.g. I use Hackers [1][2] because it is much easier to read stories on mobile and supports dark mode. Unfortunately, it doesn't support commenting or downvoting (yet?) [1] https://apps.apple.com/us/app/hackers-for-hacker-news/id6035... [2] https://github.com/weiran/Hackers

I am the developer of HACK (available on iOS, Android and MacOS) which allows commenting, upvoting, downvoting, favoriting, submitting new posts and also supports push notifications for replies. It is the top Hacker News client in the App Store.

https://apps.apple.com/ca/app/hack-for-hacker-news-developer...

Re: Hacker News Official API

#104

Earlier quoted context omitted.

I'd say it depends on how straightforward the new API will be. The app I use (Materialistic) is both free and ads-free. So I don't expect the developer to drop everything they are doing to update it if will require a lot of work. So 6 months minimum if everything is well documented, a year if it will be a hassle.

I use materialistic also, but as far as I'm aware that hasn't been an update to the app in years. In addition I believe the materialistic app listing on Google Play is also been removed. Do you have some updated version that I'm unaware of?

[deleted]

Re: Hacker News Official API

#105
post #70

In response to people's complaints about the usability of the HN Firebase API: yes. We're going to eventually have a new API that returns a simple JSON version of any HN URL. At that point we'll phase out the Firebase API, with a generous deprecation period. I'd be curious to hear people's thoughts about what a generous deprecation period might be.

The great thing about the Firebase API is that it's live, so you can subscribe to updates to a post/comment/profile and see them as they're made (no polling, just websockets). I'd hope any future version would retain this feature.

I'm reluctant to lose it as I'm about to release a browser extension that relies on it pretty heavily :)

[Edit]: this feature is undocumented in the link above, but it's a standard feature of FB and works as expected in all client libraries https://firebase.google.com/docs/database

Re: Hacker News Official API

#106

I'm currently writing an HN posts score tracker (basically it plots score variations over time since they were in new for each popular post) and it's really handy that you can just subscribe with eventsource [0] and get realtime update when something changes without having to manually poll each post wasting resources. Not yet sure about the latency though. [0] https://github.com/jpopesculian/reqwest-eventsource/

https://hnrankings.info/

Also https://upvotetracker.com/

Gives a few more variables, and the different sample times show a different set of transient effects.

Re: Hacker News Official API

#107
post #70

In response to people's complaints about the usability of the HN Firebase API: yes. We're going to eventually have a new API that returns a simple JSON version of any HN URL. At that point we'll phase out the Firebase API, with a generous deprecation period. I'd be curious to hear people's thoughts about what a generous deprecation period might be.

How difficult would it be to host the same Firebase API on top of whatever will be driving the new JSON stuff?

Yes. At the very least some kind of pub/sub interface? Also worth noting that, if it's the data depth issue that's looking to be resolved, the Algolia API is often useful in places the Firebase structure falls short.

Re: Hacker News Official API

#109

Only using the official API is quite cumbersome, as fetching a story with comments requires one network request per each comment. Instead, try considering the (more unknown) Algolia API[0]. It supports fetching the entire tree in one request. Unfortunately the comments aren’t sorted, so you’ll still have to use the official one (or parse the HN HTML) if you need that. For my HN client[1], I parse the HN HTML (as it’s…

Yeah, I built a graphql API for HN for this reason. It will let you fetch all comments, users, and posts in a single request. https://hngraphql.fly.dev/graphql

Can you share the code behind the API?

Re: Hacker News Official API

#110
post #36

> The v0 API is essentially a dump of our in-memory data structures. TIL that HN doesn't use a "proper" / persistent DB. I wonder then how HN handles data persistence. Is it as simple as creating write-ahead logs + checkpointing every some duration?

Sure they do, what they are saying is that the data format is just the raw format they use in memory after loading it from a database.

No they don’t. This forum was created as a MVP of a webapp using PG’s Arc Lisp. All user and comment data are stored as tables serialized to text files or in memory. You can find an old version of the code at http://arclanguage.org
Post reply on HN