11–20 of 43 posts
Earlier quoted context omitted.
Lots and lots of stories never make it anywhere near the front page. Mostly spam.
But I think the spam is caught and doesn't register through the api...
Awesome, this will be super useful for running ML experiments on the HN stories. Previously, Max Woolf worked on this http://minimaxir.com/2014/02/hacking-hacker-news/ via https://news.ycombinator.com/item?id=7291531
I did, and with that, I had also included that Python code in GitHub, which uses the same implementation as the OP: https://github.com/minimaxir/hacker-news-download-all-storie... Fun fact: At 10,000 calls/hour and 1,000 objects per request, you can download all stories AND comments in less than an hour. As an aside: I tried to use ML on Hacker News stories and have had exactly zero success. (i.e. the predictive mode…
https://github.com/jaredsohn/hacker-news-download-all-commen...
I have a few other things in the works related to the hnsearch API (had them almost ready for release a few months ago but then I got distracted); this post is persuading me to finish them up soon. :)
Awesome, this will be super useful for running ML experiments on the HN stories. Previously, Max Woolf worked on this http://minimaxir.com/2014/02/hacking-hacker-news/ via https://news.ycombinator.com/item?id=7291531
I'm quite excited about this data release- there are many interesting ML models that could be trained here. One I hacked on previously on my own data was a comment ranker which uses a ranking loss to rank comments consistent with their observed order in data (which roughly reflects their number of upvotes, I believe). In principle I think it could be converted to a browser extension that gives a score for how well received your comment will be conditioned on the parent comment, as you write it in the text box. One of the main issues I ran into when I hacked a bit on it was space complexity, since you need to keep all the word embeddings (usually on order of 50-200D / word) around in memory of the extension, and there are many words.
P.S. Thanks for the data, you saved a good amount of our time.
Awesome, this will be super useful for running ML experiments on the HN stories. Previously, Max Woolf worked on this http://minimaxir.com/2014/02/hacking-hacker-news/ via https://news.ycombinator.com/item?id=7291531
> Content Category: "Suspicious"
Which is an odd catch-all category. It may be a keyword match from the domain. That sucks. Anyone got the github link?
> Content Blocked (content_filter_denied) > Content Category: "Suspicious" Which is an odd catch-all category. It may be a keyword match from the domain. That sucks. Anyone got the github link?
Have you considered sharing the data as a torrent? The current FileDropper speed is quite low (~200KB/s) P.S. Thanks for the data, you saved a good amount of our time.