Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stack Exchange Machine Learning Contest

blog.stackoverflow.com

Post reply on HN
Print view View original

41–50 of 67 posts

Re: Stack Exchange Machine Learning Contest

#41

Post by bnegreve »

kmontrose wrote: post #39
m_myers wrote: post #25

Earlier quoted context omitted.

Replacing moderators with bots would be a terrible idea and hopefully will never happen. Kevin Montrose clarified it in a comment: > To be clear, we’re not intending to have an algorithm automatically close posts. This is aimed at improving our quality metrics (which guide users before posting) and auto-flagging facilities (which help focus moderation attention).

There is a 0% chance of replacing moderators with bots. I don't even know how you could read that blog post and come to that conclusion...

This: Your goal is to build a classifier that predicts whether or not a question will be closed given the question as submitted.

Re: Stack Exchange Machine Learning Contest

#42

Post by kmontrose »

bnegreve wrote: post #41
kmontrose wrote: post #39

Earlier quoted context omitted.

There is a 0% chance of replacing moderators with bots. I don't even know how you could read that blog post and come to that conclusion...

This: Your goal is to build a classifier that predicts whether or not a question will be closed given the question as submitted.

Everyone seems to have skipped right over this:

"In a perfect world, we'd be able to offer specific, targetted guidance for authors whose posts were likely to be shot down, before they ever showed up on the site, and without requiring as much up-front effort from our community."

Which is waaaaay before the classification line.

Re: Stack Exchange Machine Learning Contest

#43

Post by Homunculiheaded »

rm999 wrote: post #32
Homunculiheaded wrote: post #9

Earlier quoted context omitted.

As someone who went from top 5 to somewhere in the 60s in one contest, and reviewing results of past contests, I believe a lot of those small tweaks for slight gains in leader board scores end up penalizing the contestant for over-fitting. I saw a similar complaint to yours in a couple of forums but I do believe more often than not those small performance gains in the leader board actually hurt final scores. Addition…

It's not about small tweaks, it can be substantial additions to a model that improve its actual, out-of-sample performance. A popular method in these contests is ensembling, which involves building many sub-models and combining their scores into a single ensemble model. The netflix winner used ~100 sub-models in their ensemble, but the vast majority of the predictive power came from just three of those sub-models (ca…

Ah, I think I see what you are saying: essentially that the time it takes to build and tune the blending method and model selection for a 100+ ensemble gives you only a slightly better prediction than an appropriately choosen reasonably performant model at both a large computation and human labor cost?

What I was addressing was the issue that some users on Kaggle seemed frustrated that people were essentially submitting models with small parameter tweaks in order to marginally boost leader board scores. To these complaints I would argue that over-fitting is it's own punishment.

Thanks for the clarification!

Re: Stack Exchange Machine Learning Contest

#44

Post by rhizome »

grey-area wrote: post #29
codinghorror wrote: post #16

Earlier quoted context omitted.

> or maybe SO should just expand to encompass that sort of FAQ function? Some of the tag wikis do this: http://stackoverflow.com/tags/java/info http://stackoverflow.com/tags/c%23/info So Cletus' post might make more sense as the tag wiki for, say: http://stackoverflow.com/tags/database/info

Yes it could although the wikis don't contain the other alternative questions or the dialogue which make Q&A sessions involving several people who know the subject well useful for other readers. For many questions (particularly open ones) there are many answers, and no right answer for all circumstances, so there is no clear division between this question/answers and other longer more specific ones as to being opinio…

I have to wonder if the original plan was to leverage the question answering as a wiki generator, but if so they should put pointers on closed questions. I'm not sure whether I have a problem with how SO is conducting themselves, since it could simply be that they're opinionated about what constitutes valid content and aren't afraid to leave the other stuff to other sites. That is, I'm not sure Zawinsky's Law can be extended to GYOFB situations.

Re: Stack Exchange Machine Learning Contest

#45

Post by coolgeek »

codinghorror wrote: post #11
cletus wrote: post #3

It's nice to see this kind of contest but the topic just sets me off on a much-needed rant. The moderator situation on Stackoverflow is getting out of control. I see a Q&A site as having three main groups: 1. People who ask questions; 2. People who answer questions; and 3. People who edit/moderate questions. Even 2+ years ago there was a lot of lip service paid to the value of (3). I disagreed then and it's only been…

> But consider a question like "What are the pros and cons of Sinatra vs Rails?" This sort of question (IMHO) absolutely has value as someone experienced with both could enumerate the relative merits of each in a pretty objective fashion without making an absolute determination. This is something that absolutely could have value to anyone evaluating Ruby Web frameworks. I guess, but Zookeepers could also potentially…

> Database development mistakes made by application developers

This is a discussion, not a question. The entire text of said "question" is, quite literally, "What are common database development mistakes made by application developers?" If it can have infinite answers, is it really a question?

http://stackoverflow.com/questions/621884/database-developme....

Yes, it really is a question. I don't mean to come off as sarcastic, but it's got a question mark at the end of it - one that you, yourself, put there.

You're the one that's imposing esoteric semantics and restrictions on this.

Great post, indeed, but it belongs on your blog.

Except that:

1) no blogs have the visibility and user base that SO has - not even yours or Joel's. 2) a blog post isn't crowdsourced - at least not to the extent that SO is

One of the biggest misconceptions about Stack Exchange is this idea that discussion is, in and of itself, a net good to the world -- and therefore we are monsters for not allowing discussion. I do not believe this to be true. There is, and will always be, an infinity of discussion. Like Jay Leno once said about Doritos, "type all you want, we'll make more". If something can be had in infinite amounts, what is its value?

Nutpicking and a false dichotomy. Ease up on the defensiveness and try to see it from the point of view of the many people that want/need to know the answer to that question.

For Pete's sake, at least 568 people upvoted the question, and at least 1004 people upvoted the first answer alone.

The community has spoken - they see this as valuable content.

Stack Exchange supports only the minimal subset of discussion necessary to get practical, useful answers to specific questions. The goal is not discussion, but science-in-the-small. Back up your claims. Show us references. Show us data. Share your specific experiences.

This is an overly narrow, baffling and frustrating definition of "question"

(And the goal may not be discussion, but discussion is a characteristic of most answers. It's a community, after all.)

Re: Stack Exchange Machine Learning Contest

#46

Post by finnw »

Sounds easier than winning the Loebner Prize, and yet there is more cash on offer.

I just hope the winning entry will prompt the developers to remove that stupid filter[1] that prevents you from referring to the Halting Problem in question titles.

[1]: http://meta.stackoverflow.com/questions/107989/using-the-wor...

Re: Stack Exchange Machine Learning Contest

#47

Post by [deleted] »

usea wrote: post #37
ricardobeat wrote: post #21

probabilityOfClosing = (question) -> text = question.text.toLowerCase() return (text.length / (text.indexOf('jquery') + 2)) / 100

I realize this was in jest, but your algorithm says that the shorter a post, the less likely it is to be closed. A post of zero length has a 0% chance of being closed. Also, the result is not bound to 0..1

[deleted]

Re: Stack Exchange Machine Learning Contest

#48

Post by finnw »

MaysonL wrote: post #33
Stefan_H wrote: post #26

Earlier quoted context omitted.

You hit the nail on the head... "an appropriate forum with as many readers as SO/SE" Stack overflow on the other hand is for people to: [A]sk practical, answerable questions based on actual problems that you face. Chatty, open-ended questions diminish the usefulness of our site and push other questions off the front page.

* Chatty, open-ended questions diminish the usefulness of our site* How, pray tell?

Because they scare off the experts who post the really useful (i.e. practical) answers.

If too many of them leave, the site becomes useless.

The people who answer (and upvote) questions like "What's your favourite programmer cartoon" do not think this through - they just see something funny and click the upvote button. People have argued "but this got upvoted so it must be good" since the site started, but it just isn't true.

Re: Stack Exchange Machine Learning Contest

#49

Post by eridius »

coolgeek wrote: post #45
codinghorror wrote: post #11

Earlier quoted context omitted.

> But consider a question like "What are the pros and cons of Sinatra vs Rails?" This sort of question (IMHO) absolutely has value as someone experienced with both could enumerate the relative merits of each in a pretty objective fashion without making an absolute determination. This is something that absolutely could have value to anyone evaluating Ruby Web frameworks. I guess, but Zookeepers could also potentially…

> Database development mistakes made by application developers This is a discussion, not a question. The entire text of said "question" is, quite literally, "What are common database development mistakes made by application developers?" If it can have infinite answers, is it really a question? http://stackoverflow.com/questions/621884/database-developme... . Yes, it really is a question. I don't mean to come off as s…

For Pete's sake, at least 568 people upvoted the question, and at least 1004 people upvoted the first answer alone.

The community has spoken - they see this as valuable content.

Popular is not necessarily the same thing as valuable. This is a problem that reddit's been hitting a lot lately (e.g. image macro posts everywhere). The most easily-digested simple things get lots of upvotes because everybody can relate, or at least understand. The more complex, useful answers to more specific problems get a lot less attention because a lot fewer people can recognize the value, or even understand the question.

I have to agree with Jeff Atwood, these types of discussion questions shouldn't be on SO. They're just fluff questions that drown out the important work of solving actual problems.

Re: Stack Exchange Machine Learning Contest

#50

Post by pc86 »

Stefan_H wrote: post #26
pc86 wrote: post #20

Earlier quoted context omitted.

> I guess, but Zookeepers could also potentially talk about "What are the pros and cons of Gorillas vs Sharks?" And on an appropriate forum with as many readers as SO/SE you'd likely get well-thought out responses going far into detail regarding purchase cost, habitat maintenance, prevalence of skilled keepers, etc. Someone who actually was deciding whether to add a Gorilla enclosure or a shark aquarium would find it…

You hit the nail on the head... "an appropriate forum with as many readers as SO/SE" Stack overflow on the other hand is for people to: [A]sk practical, answerable questions based on actual problems that you face. Chatty, open-ended questions diminish the usefulness of our site and push other questions off the front page.

Thanks for the reply, Stefan, but I think you misunderstood what I meant by "appropriate" (thought I understand your use); I meant the zoological equivalent of SO/SE (Zoooverflow?)
Post reply on HN