Quick, give him a growth-hacking gig!
A Kaggle Grandmaster cheated in $25k AI contest with hidden code
81–90 of 197 posts
Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code
#82I'm not going to defend Pleskov but organizers shouldn't have put out the competition with money attached that can simply be solved by scraping data. Good ML competition in fact should even invite cheats because the end goal is not ML for the sake of ML but rather cracking the prediction problem by whatever shortest path possible.
"Winning" by anything that can reasonably called cheating, as in this case, does not advance the general state of the art. Innovation is best served through appropriate rules and competition structure.
Again, I'm not defending Pleskov. If he had come forth with the hack, things would have been different. Instead, he pretended that he had ML solution, pocket the money and put an extraordinary effort into making sure that people can't actually figure out his true doings. He was disingenuous, fully self-aware that he was in the wrong and did his best to cover up his tracks. It wasn't fair to other competitor and it was most certainly not fair to the organization trying to do something good. So yes, Pleskov, remains indefensible.
Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code
#83Earlier quoted context omitted.
You can use this to select which pets to put on a platform. For instance, no-kill shelters have to decide which animals they intake since they have finite room. They can save more animals if they pick animals that are likely to be adopted quickly. Obviously, kill shelters have a similar calculus when deciding which animals to cull (and indeed, animals that don't fit in the no-kill shelter go to the kill shelter). I'm…
I'm pretty sure so-called no-kill shelters don't outsource their killing by simply refusing less adoptable animals. And if this contest were advertised as "help us decide which animals to kill first" it probably wouldn't gain traction. This contest sounds ridiculous. It sounds like an attempt to get in on that AI gravy but do so with some sort of feel-good element. Only there is no feel good to it, and the basic prem…
> I'm pretty sure so-called no-kill shelters don't outsource their killing by simply refusing less adoptable animals.
They do, though. That's how they are able to limit the number of animals they have at any given time.
Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code
#84They would have able to win and get away with it if they incorporated the knowledge of the external dataset directly into the ML model, provided they had a reasonable estimate on the fraction of overlap between the external data and the test set. A weak version of this would be to just train on the external data in addition to the provided data. A stronger version would train regularly on the provided training data a…
"A weak version of this would be to just train on the external data in addition to the provided data."
In this competition, the training code was run on Kaggle's system, so you'd still need to smuggle in the extra data.
Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code
#85Before reading they'd scraped public data that was likely to be the "hidden" evaluation set, I thought they might have cheated using Python introspection: inspect the caller's frame, find some variable already loaded with the expected answer, return that. Has anyone cheated at Kaggle/similar using that approach?
No, and this pattern of thought is bizarre, lacks technical grounding, and more importantly, scruples
Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code
#86I'm not going to defend Pleskov but organizers shouldn't have put out the competition with money attached that can simply be solved by scraping data. Good ML competition in fact should even invite cheats because the end goal is not ML for the sake of ML but rather cracking the prediction problem by whatever shortest path possible.
No doubt he is brilliant. Many investment banks will be happy to have him. He will fit right in something like Merrill Lynch or Goldman Sachs.
He made a good career in quant finance before diving into ML and Kaggle
Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code
#87It really worries me how many people are so quick to forgive him and tell him so.
In my family if someone cheated they got called a cheater and suffered consequences. At least, they would have, if someone did something like that. But my parents didn’t raise mendacious villains.
Look at this crap on Twitter:
“Everyone makes mistakes. Thank you for the apology”.
“Kagglers will still love to have you back”
“It's great that you realize your mistakes. Looking forward to see your comeback with more cool DS solutions and ethics than before.”
“Thanks for doing this. It's okay to make errors in judgement, we've all been there to varying degrees. Y'all be gonna be fine.1!”
Those are the worst. I’m not so crazy about these below, either, although there’s just a hint of steel in them, at least:
“I’m glad to see that you had a change of heart after sleeping on it and that you will be returning the prize money. I hope you will consider donating to or volunteering at a local animal shelter as well. Atonement here is more than returning the money and apologizing.”
“I hope this can be used as a teaching moment as well. Many people clearly look up to you because of your work. What can we learn from this? Something to ponder in the days to come.”
Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code
#88Before reading they'd scraped public data that was likely to be the "hidden" evaluation set, I thought they might have cheated using Python introspection: inspect the caller's frame, find some variable already loaded with the expected answer, return that. Has anyone cheated at Kaggle/similar using that approach?
No, and this pattern of thought is bizarre, lacks technical grounding, and more importantly, scruples
The comment you're responding to isn't cracking, it's penetration testing. Hope that helps.
Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code
#89It really worries me how many people are so quick to forgive him and tell him so. In my family if someone cheated they got called a cheater and suffered consequences. At least, they would have, if someone did something like that. But my parents didn’t raise mendacious villains. Look at this crap on Twitter: “Everyone makes mistakes. Thank you for the apology”. “Kagglers will still love to have you back” “It's great t…
I’m not saying that Pleskov is a sociopath, but I’m beginning to see how sociopaths manage to make their way in the world without getting punished.
Re: A Kaggle Grandmaster cheated in $25k AI contest with hidden code
#90I'm not going to defend Pleskov but organizers shouldn't have put out the competition with money attached that can simply be solved by scraping data. Good ML competition in fact should even invite cheats because the end goal is not ML for the sake of ML but rather cracking the prediction problem by whatever shortest path possible.
No doubt he is brilliant. Many investment banks will be happy to have him. He will fit right in something like Merrill Lynch or Goldman Sachs.
>No doubt he is brilliant
How? According to the article his model without the cheat rated at ~100th place, and the article mentions him cheating the same way before (by scraping Quora for some Quora related competition).