Hacker Hacker News - see just the programming/math/science links from HN
hackerhackernews.com
Hacker Hacker News - see just the programming/math/science links from HN
1–10 of 58 posts
Re: Hacker Hacker News - see just the programming/math/science links from HN
#2I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with this; sometimes I just want to read about programming, though
I'd appreciate any feedback on this.
Re: Hacker Hacker News - see just the programming/math/science links from HN
#3I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…
Excellent work; I am worried that I am going to have to start my own social news site to get away from all the time-wasting general interest stuff here. This will save me from having to do that for a while.
Re: Hacker Hacker News - see just the programming/math/science links from HN
#4I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…
This sounds pretty awesome. Would you mind posting some of the code?
I'd be curious to know how you're formatting the input data. If you're just using the title, all of the text parsed through BeautifulSoup, some parsing algorithm to obtain the body of the article, etc.
Not too long ago, I wrote a python script to automatically extract the bodies of articles, which you may find useful: http://blog.davidziegler.net/post/122176962/a-python-script-...
Re: Hacker Hacker News - see just the programming/math/science links from HN
#5I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…
I'd love to see a threshold setting. Something like "show only above 50%" so that your visitors can adjust the visible links even further. Otherwise I love the site :)
Re: Hacker Hacker News - see just the programming/math/science links from HN
#6I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…
I worked in Bayesian spam filters for a while at the last day job. Wonderful things, aren't they?
I've got to say, your classifier looks like it is amazingly accurate for those distinctions. I'd be really interested in seeing some of the very determinative words. (There are some obvious candidates like "Erlang" and "Twitter" but every time I do a natural language processing project I'm amazed by how our distribution of "little meaningless words" changes markedly between FeatureA and FeatureB.)
Re: Hacker Hacker News - see just the programming/math/science links from HN
#7I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…
I worked in Bayesian spam filters for a while at the last day job. Wonderful things, aren't they? I've got to say, your classifier looks like it is amazingly accurate for those distinctions. I'd be really interested in seeing some of the very determinative words. (There are some obvious candidates like "Erlang" and "Twitter" but every time I do a natural language processing project I'm amazed by how our distribution…
Yeah, it's amazingly accurate for now only because the training data consists of a high percentage of the actual data set, and so it's probably classifying based on those insignificant differences. We'll see how it holds up...
Re: Hacker Hacker News - see just the programming/math/science links from HN
#8I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…
This sounds pretty awesome. Would you mind posting some of the code? I'd be curious to know how you're formatting the input data. If you're just using the title, all of the text parsed through BeautifulSoup, some parsing algorithm to obtain the body of the article, etc. Not too long ago, I wrote a python script to automatically extract the bodies of articles, which you may find useful: http://blog.davidziegler.net/po…
Thanks. I'll definitely post the code soon. I'm using the title, the submitter's name, the URL, and the page content without tags (only looking at
and similar tags would probably help). Including the content of the HN comments pages might help, too. Thanks for the link; I'll check it out.
Re: Hacker Hacker News - see just the programming/math/science links from HN
#9I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…
Why don't you provide a flag kind of button for each article that lets user train the classifier online?
Re: Hacker Hacker News - see just the programming/math/science links from HN
#10I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…
Why don't you provide a flag kind of button for each article that lets user train the classifier online?
That's a great idea. Right now it's just spitting out a static HTML file, so that'd require some significant changes to the code. I'll see how it's doing tomorrow and consider making it into something like what you describe.