Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Crowdsourcing Turned on Me

nautil.us

Post reply on HN
Print view View original

11–20 of 26 posts

Re: How Crowdsourcing Turned on Me

#11

Post by Jemaclus »

Just as an exercise in 20/20 hindsight, it seems to me that the easiest change to make is to not trust every player implicitly. In other words, you enforce some level of duplication in the work. If enough people agree that this piece goes there, then it goes there, and if not, then it doesn't. A single attacker is unlikely to be able to undo the work of dozens of supporters, even if it takes less time and effort to undermine the truth.

I believe Foursquare did something like this with regards to their recommendations. They'd ask questions about locations (e.g., pick a beautiful picture, is this place classy?, is it loud?, etc), and only when enough people agreed would they use the information in recommendations.

Re: How Crowdsourcing Turned on Me

#12

Post by Animats »

Bad design. Anybody could mess up the job. In any large, anonymous group, there will be a certain number of assholes. Of course it didn't work. Ask anyone involved in running an IRC channel, or a MMORPG, or a factory.

Given the problem they were solving (reassembling images shredded into strips) they needed something where, when two strips were put together by a human, a program decided if they matched closely, and, if so, locked them together for further editing. That would constrain the workers to make forward progress. You might need an unlock capability if there was a false match, but that should take the cooperation of several people.

Re: How Crowdsourcing Turned on Me

#13

Post by skybrian »

Animats wrote: post #12

Bad design. Anybody could mess up the job. In any large, anonymous group, there will be a certain number of assholes. Of course it didn't work. Ask anyone involved in running an IRC channel, or a MMORPG, or a factory. Given the problem they were solving (reassembling images shredded into strips) they needed something where, when two strips were put together by a human, a program decided if they matched closely, and,…

Or just versioning and the ability to revert, like Wikipedia has.

Re: How Crowdsourcing Turned on Me

#14

Post by dsuth »

GhotiFish wrote: post #5

This was a good read, but the author is not above narcissism it seems. He reminded me of the Luddites in the 19th century, who destroyed the cotton- and wool-processing technology that they feared would replace man with machine. Only in this case, the concern was reversed: The attacker seemed to dislike how crowdsourcing puts the collective potential of humans above technology. or sometimes extreme narcissism. “artis…

> In my experience, mobs like 4chan tend to descend on people who have high and mighty ideas of their actions.

In my experience, people who defend 4chan's actions like to victim-blame.

> The authors inability to empathise with "... and personally feel that crowdsourcing is basically cheating (and I’m not the only one that feels this way)." is something that makes me leery as well. That's an understandable viewpoint, how is this going to work with actually classified documents?

Actually this line (from the attacker) garnered the most attention from me. So the attacker felt that crowd-sourcing the solution was cheating - but maliciously attacking and sabotaging an opponent in a competition was just fine? This sense of entitlement is very typical of anti-social b-tard behaviour, and it shouldn't be encouraged.

Personally I find the tension between clever algorithms and crowd-sourcing very interesting. Applications like Foldit for protein folding produce some amazing results. Sabotaging valid solutions does nobody any favours, except in an extremely selfish and short-term way.

Re: How Crowdsourcing Turned on Me

#15

Post by xerophyte12932 »

incision wrote: post #2

Fascinating stuff, not at all what I was expecting. Two things come to mind reading this. 1.) The crowdsourced and algorithmic solutions seem to be framed in contrast if not exclusive to each other here. I wonder if combining the two couldn't have helped avoid this: >'However, the crowd was hopeless against a determined attacker. Before the first attack, our progress on the fourth puzzle had combined 39,299 moves by…

> The sort self-importance expressed by the attacker is just awful

While I agree that instead of behaving maliciously, the other competitor should have lauded their simple yet effective solution, I can imagine the WTF he must have experienced upon discovering their site. I mean think about it, if your team has been spending god knows how long wracking your brains to come up with CV algorithms to solve this stuff, and then you realize the team in the lead isn't even using an algorithm! They are just using a huge amount of people to solve it! Now of course the right reaction to this should be to laugh wholeheartedly and ponder about how a motivated crowd can trump advanced technology.

We had a similar WTF moment during Google Code jam 2011. In the qualification rounds, there was a problem Goro Sort[1] and the way the question was written, it appeared to be a very tough problem. Later we discovered that one of our friends solved it with a VERY simple algorithm: just count the number of elements out of place! and it WORKS!! The good part was, instead of being jealous, we did laugh about it and still do

Re: How Crowdsourcing Turned on Me

#16

Post by xerophyte12932 »

Animats wrote: post #12

Bad design. Anybody could mess up the job. In any large, anonymous group, there will be a certain number of assholes. Of course it didn't work. Ask anyone involved in running an IRC channel, or a MMORPG, or a factory. Given the problem they were solving (reassembling images shredded into strips) they needed something where, when two strips were put together by a human, a program decided if they matched closely, and,…

>In any large, anonymous group, there will be a certain number of assholes. Of course it didn't work. Ask anyone involved in running an IRC channel, or an MMORPG, or a factory.

My favourite example: Twitch plays Pokemon

Re: How Crowdsourcing Turned on Me

#17

Post by xerophyte12932 »

GhotiFish wrote: post #5

This was a good read, but the author is not above narcissism it seems. He reminded me of the Luddites in the 19th century, who destroyed the cotton- and wool-processing technology that they feared would replace man with machine. Only in this case, the concern was reversed: The attacker seemed to dislike how crowdsourcing puts the collective potential of humans above technology. or sometimes extreme narcissism. “artis…

I don't think Darpa was looking for an algorithm to unshred documents, as the article states, they were assessing vulnerability of shredded documents. In that regard, this team's efforts were most important as they highlighted how an adversary doesn't even need a complex CV algorithm, all they need is a huge crowd and an incentive

Re: How Crowdsourcing Turned on Me

#18

Post by xxs »

Animats wrote: post #12

Bad design. Anybody could mess up the job. In any large, anonymous group, there will be a certain number of assholes. Of course it didn't work. Ask anyone involved in running an IRC channel, or a MMORPG, or a factory. Given the problem they were solving (reassembling images shredded into strips) they needed something where, when two strips were put together by a human, a program decided if they matched closely, and,…

if they could do the matching they would have not needed the human force.

Re: How Crowdsourcing Turned on Me

#19

Post by xxs »

skybrian wrote: post #13
Animats wrote: post #12

Bad design. Anybody could mess up the job. In any large, anonymous group, there will be a certain number of assholes. Of course it didn't work. Ask anyone involved in running an IRC channel, or a MMORPG, or a factory. Given the problem they were solving (reassembling images shredded into strips) they needed something where, when two strips were put together by a human, a program decided if they matched closely, and,…

Or just versioning and the ability to revert, like Wikipedia has.

They would need huge amounts of versions and then need someone manually to observe. With a single attacker would be easy, though.

Re: How Crowdsourcing Turned on Me

#20

Post by xxs »

GhotiFish wrote: post #5

This was a good read, but the author is not above narcissism it seems. He reminded me of the Luddites in the 19th century, who destroyed the cotton- and wool-processing technology that they feared would replace man with machine. Only in this case, the concern was reversed: The attacker seemed to dislike how crowdsourcing puts the collective potential of humans above technology. or sometimes extreme narcissism. “artis…

That's an understandable viewpoint, how is this going to work with actually classified documents?

My thoughts exactly - the system is designed for the military, now imagine getting thousands of random people peeking at the documents.

Post reply on HN