Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacker Hacker News - see just the programming/math/science links from HN

hackerhackernews.com

Post reply on HN
Print view View original

11–20 of 58 posts

Re: Hacker Hacker News - see just the programming/math/science links from HN

#11

Post by iamelgringo »

sqs wrote: post #10
paraschopra wrote: post #9

Earlier quoted context omitted.

Why don't you provide a flag kind of button for each article that lets user train the classifier online?

That's a great idea. Right now it's just spitting out a static HTML file, so that'd require some significant changes to the code. I'll see how it's doing tomorrow and consider making it into something like what you describe.

Are you using the reverend thomas library? http://www.divmod.org/trac/wiki/DivmodReverend

I've been wanting to use that library for something just like this. I would love to read a write up of this, or see the code at some point.

Thanks

Re: Hacker Hacker News - see just the programming/math/science links from HN

#12

Post by ersi »

jrockway wrote: post #3
sqs wrote: post #2

I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…

Excellent work; I am worried that I am going to have to start my own social news site to get away from all the time-wasting general interest stuff here. This will save me from having to do that for a while.

That doesn't sound very social. Aren't most programming/math/science also "time-wasting", so to speak?

Re: Hacker Hacker News - see just the programming/math/science links from HN

#13

Post by magv »

sqs wrote: post #2

I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…

> Page generated automatically at 23:49 on 05 July 2009

Can you specify the timezone?

Also, how often is the page updated?

Re: Hacker Hacker News - see just the programming/math/science links from HN

#14

Post by markbao »

sqs wrote: post #2

I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…

Now we need Startup Hacker News.

Re: Hacker Hacker News - see just the programming/math/science links from HN

#16

Post by 10ren »

sqs wrote: post #7
patio11 wrote: post #6

Earlier quoted context omitted.

I worked in Bayesian spam filters for a while at the last day job. Wonderful things, aren't they? I've got to say, your classifier looks like it is amazingly accurate for those distinctions. I'd be really interested in seeing some of the very determinative words. (There are some obvious candidates like "Erlang" and "Twitter" but every time I do a natural language processing project I'm amazed by how our distribution…

Yeah, it's amazingly accurate for now only because the training data consists of a high percentage of the actual data set, and so it's probably classifying based on those insignificant differences. We'll see how it holds up...

I think you're training it on the submission titles - I wonder if the text of the websites themselves might be more accurate. Certainly richer. But it's quite possible that the submission titles are more accurate.

Re: Hacker Hacker News - see just the programming/math/science links from HN

#17

Post by alexgartrell »

The reason hacker news sucks less than other places is cause when people come up with cool mashups that could be interpreted negatively (you don't like all the news chosen by the community, whaaatttt?!), no one gets pissy or flamey.

It's almost like "the way society should work" or something...

Re: Hacker Hacker News - see just the programming/math/science links from HN

#20

Post by jlees »

markbao wrote: post #14
sqs wrote: post #2

I made this in a few hours for my own personal use and thought that other people might want to use it, too. It uses a Python Bayesian classifier trained on about 200 hand-labeled links and outputs a static HTML file. It's not perfect (and the technical/non-technical distinction is arbitrary), but I'll give it some more training data and see if it improves. Also, I don't mean to disparage non-technical topics with thi…

Now we need Startup Hacker News.

What about a generic My Hacker News with a classifier trained on each person's personal "I want to see / I do not want to see" preferences? :>
Post reply on HN