Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Kaggle Grandmaster cheated in $25k AI contest with hidden code

theregister.co.uk

Post reply on HN
Print view View original

51–60 of 197 posts

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#51

Post by geomark »

guelo wrote: post #21
nefitty wrote: post #5

Earlier quoted context omitted.

It’s so fascinating to see how easily a human brain can be hijacked by status. I wonder if there’s a way to harness that power for improving individual performance...

Elementary shchool teachers use it all the time. Just some stickers and a leaderboard in the classroom and it works wonders for behavior and achievement.

I have noticed that it doesn't have much effect after a while. The students are like, meh, someone gets an award every week, and eventually it rolls around to my turn unless I really screw up.

The effect on parents seems bigger, which is a good thing, I suppose, since parents then give their kids to some positive feedback.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#52

Post by oefrha »

floatingatoll wrote: post #4

> " For me, it was never about the money but rather about the Kaggle points " Accumulated integer counts that represent social value and/or standing encourage destructive behavior.

> Be very, very careful when you put a number next to someone's name. Because people will do whatever it takes to make that number go up.

Jeff Atwood on Coding Horror. (He can't be the first person to have said this, though.)

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#53

Post by MiroF »

blurps wrote: post #29
quickthrower2 wrote: post #26

Wouldn’t it be smarter to use the hidden data as training data rather than hard code it in?

This only works when you don't win. You have to upload your code, including model training code, they won't accept a trained model binary.

If they were smart, they would have used the whole set for hyperparameter tuning. That would be essentially undetectable.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#54

Post by MiroF »

harry8 wrote: post #36

tl;dr Competive ML model grading with a common training set using unseen data. Cheat was to scrape the data that would then be used as unseen by the organisers. Unseen is now seen for this model. Then, instead of training the model with the "unseen" data, which would have been cheating and an advantage it apparently wasn't enough of an advantage so they hard code 10% of the cases to boost metrics and win. Having more…

They train winning code from scratch, I think, so you wouldn't have been able to just train the model with the "unseen" data.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#55

Post by marta_morena »

gdsdfe wrote: post #10
nefitty wrote: post #5

Earlier quoted context omitted.

It’s so fascinating to see how easily a human brain can be hijacked by status. I wonder if there’s a way to harness that power for improving individual performance...

There is and it's very effective ... A company I worked in briefly used to have this badge system to honor people that go the extra mile at work, and people were very keen on getting those even though it didn't have any financial compensation behind it

Um that is kindof missing the point. A badge is nothing more than a number expressed in symbols, so you demonstrated just another variation of how to hijack the brain, even if you just read an article showing it to you with numbers...

If "improving individual performance" means to you that you perform better at your job, you are already lost.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#56

Post by yborg »

oefrha wrote: post #52
floatingatoll wrote: post #4

> " For me, it was never about the money but rather about the Kaggle points " Accumulated integer counts that represent social value and/or standing encourage destructive behavior.

> Be very, very careful when you put a number next to someone's name. Because people will do whatever it takes to make that number go up. Jeff Atwood on Coding Horror. (He can't be the first person to have said this, though.)

Unless it's their weight.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#57

Post by Polylactic_acid »

geomark wrote: post #51
guelo wrote: post #21

Earlier quoted context omitted.

Elementary shchool teachers use it all the time. Just some stickers and a leaderboard in the classroom and it works wonders for behavior and achievement.

I have noticed that it doesn't have much effect after a while. The students are like, meh, someone gets an award every week, and eventually it rolls around to my turn unless I really screw up. The effect on parents seems bigger, which is a good thing, I suppose, since parents then give their kids to some positive feedback.

Thats only if the points are simply participation awards. They only have value if winning them is not for granted.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#59

Post by sytelus »

I'm not going to defend Pleskov but organizers shouldn't have put out the competition with money attached that can simply be solved by scraping data. Good ML competition in fact should even invite cheats because the end goal is not ML for the sake of ML but rather cracking the prediction problem by whatever shortest path possible.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#60

Post by jliptzin »

yborg wrote: post #56
oefrha wrote: post #52

Earlier quoted context omitted.

> Be very, very careful when you put a number next to someone's name. Because people will do whatever it takes to make that number go up. Jeff Atwood on Coding Horror. (He can't be the first person to have said this, though.)

Unless it's their weight.

You’d be surprised
Post reply on HN