Live data from Hacker News

Hacker News API

blog.ycombinator.com

121–130 of 315 posts

Re: Hacker News API

#123
post #55

Earlier quoted context omitted.

'Simple site' that gets probably tens of thousand if not hundreds of throusand unique visitors a day.

HN uses 12 lines of javascript total, it's pretty damn simple.

And what about the backend? Last I heard it's all stored as files on a handful of machines, so it's not like it's some off the shelf database you can just keep throwing machines at to scale. Also it's written in lisp originally by one person, so have fun figuring out how it all works and ramping up to it quickly.

There's also certainly a big support overhead dealing with the moderation of stories and comments.

I'm not saying it should take hundreds of dedicated engineers to run the site, but I think it's silly to look at it and think you could run it in your spare time because it's just a list of stories.

Re: Hacker News API

#124

Disappointing they are still going to use Arc even with the update to modern HTML. I despise how with Arc if I leave the page open during the day and try to click a link later the link has expired. They should really update to a modern web framework at the same time. Big modern frameworks like Rails are making ridiculously awesome improvements like replacing page loads with XHR (quicker loading since JS/styles/etc is…

What is it about Arc that causes the unexpired link issue? I know very little about Arc. But I understand it to be a language whereas the issue you describe appears to be the function of some implementation decision. Can you please elaborate?

(I'll take a crack at explaining this momentarily...)

Good grief, that took a long time. Here you go: http://pastebin.com/bSW5dfRQ [1]. I'd better stop neglecting my duties now!

Edit: One thing I forgot to put in there: one reason the closure technique is powerful is that you're leveraging the programming language and runtime to do most of the book-keeping for you. Whatever data is handy, you just reference. The system will remember all the references. That's why using things like query strings and hidden form fields is more complicated: you have to handle all those details yourself (not to mention serialize and deserialize them if you're passing through any other format than what your program keeps in memory). That is tedious, and when your app has many kinds of request, the complexity quickly piles up.

Of course there are other abstractions you can build over this, but closures are an elegant one—especially in cases where programming simplicity is more important than scalability, which is most cases.

Edit 2: A few people thought this should be its own post, so I made https://news.ycombinator.com/item?id=8425011.

[1] Originally http://pastebin.com/dETyYtpX, but I added the above bit etc.

Re: Hacker News API

#125

This... is cool, but also kinda sucks for me. I've invested dozens of hours into writing an extremely complicated scraper for my Android version of HN. https://play.google.com/store/apps/details?id=com.airlocksof... The newest version (still under development, probably a month or two from release) adds support for displaying polls, linking to subthreads, and full write support (voting, commenting, submitting, etc). I…

I'm also currently writing a scraper[1] for the HN frontpage (for my WIP Hacker News redesign), and while there's a limited Algolia API available, it doesn't do much good if users can't post comments, upvote etc. Same goes for the official one now. So, @anyone involved with the API project, can you give us an estimate on when will the OAuth-based user-specific API be rolled out? I'm fining with pausing my efforts unt…

It's not going to be soon :(

Re: Hacker News API

#126

Oh man you guys, patio11 has generated massive amounts of content: https://hacker-news.firebaseio.com/v0/user/patio11.json?prin... I count 8,483 submissions. I'm sure there's something interesting to be done with all of this data. A word frequency chart? --- Edit: So apparently there's a ruby gem that lets you feed it a body of text and generates pseudo-random phrases based on that text. I present to you the patio11…

In February, I did an analysis on all Hacker News stories: http://minimaxir.com/2014/02/hacking-hacker-news/ Yesterday, I published an analysis on all Hacker News comments: http://minimaxir.com/2014/10/hn-comments-about-comments/ There's a lot of interesting trends in the data. Let me know if you want to know anything in particular and I'll get back to you. :)

Thanks for that. Great stuff.

Re: Hacker News API

#127

This... is cool, but also kinda sucks for me. I've invested dozens of hours into writing an extremely complicated scraper for my Android version of HN. https://play.google.com/store/apps/details?id=com.airlocksof... The newest version (still under development, probably a month or two from release) adds support for displaying polls, linking to subthreads, and full write support (voting, commenting, submitting, etc). I…

I did use your app for learning purposes - I studied the code quite a lot when learning Android. Thanks for good job !

Re: Hacker News API

#128
post #104

I am excited about this because now I can finally build a way to query my own posts and comments. I often times come across products/services/cool hacks on here that I vaguely remember, but cannot always locate. Using built-in search is kludgy, and I'd like to be able to do something more complex. Thanks for doing this!

It's interesting that you mention the phenomenom of "I remember seeing it (or maybe even saying it) but I cannot find it". This has happened to me a lot. It is intensely frustrating to me because it's blocking - I don't get much done until I re-find whatever it was. I wonder how many other people have this? And what their techniques are?

I have the same problem, and I think the best solution would be to index my web browser search history and be able to search through that index. However that would probably generate a lot of junk. My current solution is to bookmark anything that's vaguely interesting. And when I remember something, I search through the bookmarks. I never look at them, they're just for keeping the stack of stuff to search through.

Re: Hacker News API

#130

Disappointing they are still going to use Arc even with the update to modern HTML. I despise how with Arc if I leave the page open during the day and try to click a link later the link has expired. They should really update to a modern web framework at the same time. Big modern frameworks like Rails are making ridiculously awesome improvements like replacing page loads with XHR (quicker loading since JS/styles/etc is…

I guess you don't use HN very often. We fixed that in nearly all cases a few months ago :)
Post reply on HN