Facebook’s Hacker Cup Called A Failure

Yeah FB Hacker Cup surely had a lot of issues as pointed out. The thing which happened to me was even more surprising. I participated in Qualification Round, and received confirmations mails(2) as well. Later, I was able to see myself in Qualification Scoreboard. But on the day of round 1A it stopped showing me as qualified to round 1A.

The article is just about the qualification round.

The first post-qualification round was even worse (I was a competitor).

I won't go through all the details, but problems included mis-specified questions.

The second question - a 2 variable optimisation problem, set as a Starcraft game problem, in which shield generators cost X and warriors cost Y, and you had some rules about the cost/benefits of each - asked you to report the number of warriors to build, to maximise damage done.

Writing code to solve this, it quickly became clear that the sample answers provided were wrong.

Mid way through the competition, they changed the question to require you to output the number of shields to build - which, of course, made the sample output make sense. While they put up a post mid competition saying they suspected a bug in the question, they did not subsequently make any communication that they had changed the question, or confirm the error. No text was added to the question page notifying competitors it had changed. No confirmation of the change was officially made, afterwards. (You can verify this independent of my comment by reading the competition walls, or googling - the question was called 'power overwhelming').

Another issue - while they were technically correct - was where they said input to the first question would be whitespace separated, and give training examples showing it space separated, but then the test problems were Newline separated. While its debatable whether Newline is whitespace (I'd say it is, but it typically doesn't match \s) this change from the training examples to the test confused a lot of people - and its bad programming competition form, to trip people up in the input specification.

Finally, due to the slew of problems in the round, they closed the round early, without warning. This upset competitors who came to submit their solutions, to find that the round had already closed.

AFAIK, they still have not clarified whether competitors from that round that answered questions will progress, or will all have to redo it.

Communication - including on topics such as rules - has been sporadic, and either via e-mail, or through posts left on the wall - sometimes in comments on the competition wall, rather than in organised FAQs, or using the facebook notification system.

Answer upload has also been temperamental for certain browsers.

But posing a programming competition question to 1000s of competitors, where the actual question is mis-specified - that's a very big error.

Rescheduling the succeeding sub-rounds at the last minute also inconvenienced a lot of programmers who had taken time out of their schedules to do it.

Its a programming competition, and its just for fun - but I'd like to think they'd do a better job considering the volume of contestants, and the wide external audience.

The main problem wasn't the fact that rules were not stated correctly or that people didn't read them (at least when i started it, it was clear that you just had 6 minutes to generate the output after downloading the input file, make sense imho), it was the shoddy implementation of the site and lack of care on things like test output (missing lines, typos...). I did it on a sunday afternoon, not having anything better to do, but just the fact that when uploading the results the top list didn't get updated... was chrome fault? was my output for those dead-easy problems wrong? Cmon, this issues should have been caught during the testing phase of this webapp. Next time, i suggest to just open a contest on topcoder...

Certainly wasn't a simple hackercup. I participated in both rounds so far, the first one was a mess, I tripped off the 6 minute timer reviewing the questions on my iPad in the morning, without even downloading the input files! Maybe I should have read the enourmos set of instructions before, but I did it for the fun, and reading long t&cs isnt my idea of weekend fun.

The next round was even worse, thankfully I spotted pretty quickly no one was solving the starcraft problem, an obvious problem to attract starcraft fans, and there are enough of those in the programming community. So gave that a miss and moved onto the others. Looks like most people got caught out on that problem, which was a real shame. They need to reset this round and start again to be fair, but I'm sure given how upset a lot of competitors were the server load won't be the same next time.

Yeah I feel your pain. I submitted a question. Time expired and after the round closed, it said I hadn't submitted anything and thus I am disqualified.