Live data from Hacker News

A Kaggle Grandmaster cheated in $25k AI contest with hidden code

theregister.co.uk

61–70 of 197 posts

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#61

> The goal was to create an algorithm that could predict how quickly a pet would be adopted based on its profile details, from its photo to its breed, sex, size, age, and whether it had been vaccinated or not. > These predictions would be used to optimize and tweak future critters' profiles so that they are adopted as soon as possible. Sorry but, how is this useful? You can't just change the age of an animal to make…

The purpose was eugenics, on animals this time around.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#62
post #54
post #36

tl;dr Competive ML model grading with a common training set using unseen data. Cheat was to scrape the data that would then be used as unseen by the organisers. Unseen is now seen for this model. Then, instead of training the model with the "unseen" data, which would have been cheating and an advantage it apparently wasn't enough of an advantage so they hard code 10% of the cases to boost metrics and win. Having more…

They train winning code from scratch, I think, so you wouldn't have been able to just train the model with the "unseen" data.

They scraped data from the website of the org that wanted the results and funded the competition. That data was supposed to be unseen for the competitors and used to grade the models. This cheat was to use that scraped data in its training set and, beyond that, hard code some predictions.

It's looking up the answers in the grading sheet while taking an exam.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#63
post #59

I'm not going to defend Pleskov but organizers shouldn't have put out the competition with money attached that can simply be solved by scraping data. Good ML competition in fact should even invite cheats because the end goal is not ML for the sake of ML but rather cracking the prediction problem by whatever shortest path possible.

No doubt he is brilliant. Many investment banks will be happy to have him. He will fit right in something like Merrill Lynch or Goldman Sachs.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#64

Earlier quoted context omitted.

Propose HN: 'Countless' experiment – hiding all karmic integers : https://news.ycombinator.com/item?id=19745267 Perhaps someday we'll get a week without them.

No chance. HN’s goal is to have every effective programmer in the world checking it at least once a week. Anything that gets in the way of that is de facto bad. (I’d love to be surprised, though.) You can get the same effect with a Tampermonkey script I wrote: https://news.ycombinator.com/item?id=14456200 It even scrambles the karma count in your profile, so there’s no opportunity for karma to affect you.

The proposal is to see if site behavior changes as a whole when integers are removed. Individual opt-in doesn't permit such a test, nor does permitting opt-out.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#65
post #59

I'm not going to defend Pleskov but organizers shouldn't have put out the competition with money attached that can simply be solved by scraping data. Good ML competition in fact should even invite cheats because the end goal is not ML for the sake of ML but rather cracking the prediction problem by whatever shortest path possible.

"Winning" by anything that can reasonably called cheating, as in this case, does not advance the general state of the art. Innovation is best served through appropriate rules and competition structure.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#66
post #45

> The goal was to create an algorithm that could predict how quickly a pet would be adopted based on its profile details, from its photo to its breed, sex, size, age, and whether it had been vaccinated or not. > These predictions would be used to optimize and tweak future critters' profiles so that they are adopted as soon as possible. Sorry but, how is this useful? You can't just change the age of an animal to make…

You can use this to select which pets to put on a platform. For instance, no-kill shelters have to decide which animals they intake since they have finite room. They can save more animals if they pick animals that are likely to be adopted quickly. Obviously, kill shelters have a similar calculus when deciding which animals to cull (and indeed, animals that don't fit in the no-kill shelter go to the kill shelter). I'm…

I'm pretty sure so-called no-kill shelters don't outsource their killing by simply refusing less adoptable animals. And if this contest were advertised as "help us decide which animals to kill first" it probably wouldn't gain traction.

This contest sounds ridiculous. It sounds like an attempt to get in on that AI gravy but do so with some sort of feel-good element. Only there is no feel good to it, and the basic premise seems outlandish.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#67

Earlier quoted context omitted.

Propose HN: 'Countless' experiment – hiding all karmic integers : https://news.ycombinator.com/item?id=19745267 Perhaps someday we'll get a week without them.

No chance. HN’s goal is to have every effective programmer in the world checking it at least once a week. Anything that gets in the way of that is de facto bad. (I’d love to be surprised, though.) You can get the same effect with a Tampermonkey script I wrote: https://news.ycombinator.com/item?id=14456200 It even scrambles the karma count in your profile, so there’s no opportunity for karma to affect you.

[deleted]

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#68
post #51
post #21

Earlier quoted context omitted.

Elementary shchool teachers use it all the time. Just some stickers and a leaderboard in the classroom and it works wonders for behavior and achievement.

I have noticed that it doesn't have much effect after a while. The students are like, meh, someone gets an award every week, and eventually it rolls around to my turn unless I really screw up. The effect on parents seems bigger, which is a good thing, I suppose, since parents then give their kids to some positive feedback.

For monthly things, it does have some effect.

I participated in some of them before due to the visibility. People from your class may know you but others, not so much. In the absence of time and other signals, management would often pick someone who is highlighted to represent something.

A lot of big people at those events generally don't mind giving contact information to young kids asking for help as much.

From there, you can effectively build a nice list of people that you can leverage in future for your resume or career.

It's like free consultation you otherwise would need to buy later.

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#69

Earlier quoted context omitted.

Propose HN: 'Countless' experiment – hiding all karmic integers : https://news.ycombinator.com/item?id=19745267 Perhaps someday we'll get a week without them.

No chance. HN’s goal is to have every effective programmer in the world checking it at least once a week. Anything that gets in the way of that is de facto bad. (I’d love to be surprised, though.) You can get the same effect with a Tampermonkey script I wrote: https://news.ycombinator.com/item?id=14456200 It even scrambles the karma count in your profile, so there’s no opportunity for karma to affect you.

[deleted]

Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code

#70
post #3

Never understood Kaggle. The Netflix Prize was great. Now it’s just people gaming Kaggle to get a job.

The Netflix Prize was interesting, and drew a lot of attention to it, but ultimately wasn't that simply stuffed into the trash bin? Not long after that, Netflix basically abandoned both user ratings of significance, and realistic recommendations. Now it's just a nonsense engine with some sort of meaningless overlap or whatever they call it.
Post reply on HN