Live data from Hacker News

Gamification is not entirely bullshit

22ideastreet.com

21–30 of 35 posts

Re: Gamification is not entirely bullshit

#21
post #19

Gamification is bullshit by design. Gameful mechanics may not be, but we're not there yet. Point 1 is that Zichermann (who wrote Games-Based Marketing) misrepresents the research into motivation. Whether this is maliciousness or incompetence is up for debate (personally I think the latter with a dose of arrogance). You can see Deturding shred his O'Reilly book, where he goes into this in a lot of detail [1]. One poin…

Seeing as you seem to be well-read in the area - what's your opinion of the "gamification" on khanacademy?

The reason I've been reading it all is because I'm writing my thesis on it :)

I haven't spent too much time with Khan Academy, so can't say too much about it. There are two things to keep in mind:

1. Extrinsic motivation erodes intrinsic motivation. The most obvious example is asking a kid to take out the trash. Once you pay him/her, they'll only do it for money again. While this example is silly because no-one has any intrinsic motivation to take out the trash, it gets really dicey when you add rewards to things like reading. Are they reading to develop a love of reading, or for the carrot you put at the end? What happens when the carrot goes away? The research suggests that once the carrot disappears, so does the reading. The psychology isn't completely cut-and-dried on this, as no psychology seems to be, but it's compelling to me, at least.

The problem is that some scholars, particularly Alfie Kohn, would say that things like gold stars are rewards, and thus you're eroding things that way. However, you have to balance that with providing legitimate feedback for knowing how well you're doing, as people want to feel competent. Both these come from self-determination theory (read up on it, it's neat). The trick is to make sure the reward doesn't try to control behavior, but rather provide feedback on behavior. eg. "you did x 5 times" (good) vs "you did x 5 times as required" (bad). It's a really fine line.

As far as I can see, the main goals in Khan Academy seem to be informational and are good.

2. Badges are game systems themselves (I heard this from Nicole Lazzaro [1]), and they will result in increasing behavior of certain things for people that pay attention. They do not necessarily result in increased behavior of value. eg. a badge for posting 5 times is earned whether your posts are insightful, or whether they simply say "first post". And, again, these are extrinsic rewards, and once they dry up, people stop caring. There's also a strong possibility that rewards suffer from inflation, and you have to hit people with bigger and bigger rewards once they start relying on them.

The badges at Khan Academy make me wary [2]. Check out this one: "Quickly & correctly answer five questions in a row." What's the goal here? Khan Academy's goal, one would think, is to encourage students to think deeply about problems and arrive to correct solutions. It spends a lot of time saying students are able to work at their own pace, and this is a benefit, but then we have a badge which encourages quick, possibly slap-dash, behavior. It's more controlling than informational, requesting the student do something not necessarily natural to him/her in order to achieve it.

The ones given out for achieving proficiency are better; they align with what they want students to do and provide informational feedback that students have done it, increasing their motivation.

====

Now, this is not to say that all achievement systems are bad or doesn't work. Stack Overflow's seems to do really well. If it was doing poorly, we'd expect a large drop off of people when the extrinsic rewards dry up, largely at reputation 10,000, when you get moderator tools. The current data from Stack Overflow indicates only about 5% or so of users drop off once they get the tools (I can dig out references if you're interested). Upvotes and achievements provide useful feedback about your mastery of SO, so with that low attrition rate, it indicates they are doing things right.

The elephant in the room that Stack Overflow presents is that it may indicate that gamification does work in its entirety, and that all this motivation theory is invalid in this domain for some reason. This is possible, but such a strong statement would require much harder evidence than we have. Right now, it's more plausible that Stack Overflow is simply doing things well and seems to be aligning with the recommendations of the research, rather than indicating a wider problem with the applicability of this stuff.

[1] http://www.xeodesign.com/founder.html [2] http://www.khanacademy.org/badges

Re: Gamification is not entirely bullshit

#22

The most bizarre bit of gamification I've run across lately was in Wells Fargo's ATMs. I deposited a check the other day and got an alert on the ATM screen: You've earned a new badge Express Depositor [Learn More] Well! That should certainly help me gain status and reputation among my fellow ATM users! Photo here: http://artlung.com/blog/2012/05/03/gamification-in-the-wild-...

Oh goodness.

Have you seen this at any other Wells Fargo ATMs? Do you know if there's a web interface? The only mention I can find of this is the linked image.

Re: Gamification is not entirely bullshit

#23
post #19

Earlier quoted context omitted.

Seeing as you seem to be well-read in the area - what's your opinion of the "gamification" on khanacademy?

The reason I've been reading it all is because I'm writing my thesis on it :) I haven't spent too much time with Khan Academy, so can't say too much about it. There are two things to keep in mind: 1. Extrinsic motivation erodes intrinsic motivation. The most obvious example is asking a kid to take out the trash. Once you pay him/her, they'll only do it for money again. While this example is silly because no-one has a…

Thank you for your in depth and insightful reply - what you say makes a lot of sense. Good luck with your thesis and I would personally love to see a summary of your findings posted here on HN :)

Re: Gamification is not entirely bullshit

#24
post #12
post #9

Earlier quoted context omitted.

I believe this phenomenon it's characteristic of highly interdiciplinary areas, mostly practical and technical but with a strong component of humanism. The people that want to do work in industry tend to gravitate to some specific fields. And half a decade in managment or engineering school can't teach you the history of philosophy, anthropology, sociology or psychology. I believe this to be the reason why "design" i…

I think that's definitely part of it, but I think there may be a bit of deliberate amnesia as well, out of a desire to portray something as totally new , and perhaps to avoid any unwanted historical baggage. You also see this in "serious games" and its variants. Even Al Gore was recently talking about how the future of videogames is that maybe they'll be used for things like education, too, not just entertainment. Su…

That's true, the process doesn't need to be 100% "conscious".

PS: Great insight on the article about soviet "gamification", I never thought abut that in that way, thanks.

Re: Gamification is not entirely bullshit

#25

Earlier quoted context omitted.

I up-voted you because I like you directly teasing out this point. This is what I meant by flipping the script. It's just a matter of perception, and that itself can be changed. It shifts with culture. If you think work is a chore then it will be a chore. If you think it is fun then it will be that, at least some of the time. If innovations come around that make it more fun regardless, culture will shift where less w…

For the vast majority of people work is always a chore. It's a commodity you use to barter with your employer; in exchange you receive money that allows you to live. You argue that sometimes work is "fun". I think a more accurate description is that work can sometimes be engaging. Like when you're presented with a problem that's right at the edge of your abilities but not out of reach. Yet it's still work. Unlike pla…

Play is a frame of mind, not a contractual relationship. If it weren't, nobody would take mmorpgs seriously, and they do; the consequences may not involve your paycheck but it's harsh when you let your team down.

Playfulness shows up at work all the time, although not necessarily in ways that promote the bottom line.

Play mindsets can be encouraged in the workplace when they don't emerge spontaneously. Whether those efforts are successful for a given person on any given day...? It's still pushing a rock uphill.

Game design and reward systems are trying to find a path that doesn't suck. It may take another decade of experimentation before ten thousand failed attempts show the obvious and elegant ways to make more workplace leaders look like your favorite camp counsellor, recess organizer, or dungeon master.

Re: Gamification is not entirely bullshit

#26

Gamification is bullshit by design. Gameful mechanics may not be, but we're not there yet. Point 1 is that Zichermann (who wrote Games-Based Marketing) misrepresents the research into motivation. Whether this is maliciousness or incompetence is up for debate (personally I think the latter with a dose of arrogance). You can see Deturding shred his O'Reilly book, where he goes into this in a lot of detail [1]. One poin…

Too late for an edit, but I wanted to note that this post does make mention of quality of interaction, I just forgot by the time I wrote the passage.

The first rater of a deal would get a bunch of points regardless of their rating of a deal, and so their rating didn’t matter. They would just spam (in the gamer-speak sense) the “10″ rating since it was closest to the next deal button, and do this hundreds of times without looking at the deal at all. Future voters would get points based on the crowd average, so the first voter would skew the points rating all of the deals in one sitting. It ended up that the ratings on the deals were not really accurate, thus undermining the crowdsourcing of the deals and thwarting our business goals. However, this example was more of an implementation failure than a failure of gamification in general.

I think the OP is too hard on himself here. It's very hard to imagine a game mechanic which would elicit the feedback you want: namely that you want people to give an honest opinion of something. How do you place value on an opinion? How can you tie in something so subjective to a mastery goal?

I was placed on a very similar task as part of some consulting I've been doing. I haven't found a solution, and I don't think a good one exists. The best I could come up with was that users had two interactions, not one. The first one was their honest opinion, the second one was a guess on what the prevailing opinion was. The OP only did the second, which is only going to result in a very quick convergence on groupthink. This was rejected by the company, as they didn't want to have two interactions instead of one, which is understandable. I sometimes think they truly believe that what they want is just waiting to be discovered, and I'm just not doing a good job of it.

The OP's passage highlights one of the most insidious aspects of gameification that I think often goes unsaid: it encourages you to find a holy land that may not exist, and in trying to get there, you accidentally do active harm to your goals. You lead your business down the road to hell with your gameification intentions, and your thinking gets so totally focused on this you don't see the other, probably better, options available. FWIW, I think the very same thing can be said about any company that uses the word "social" without actually backing it up with a real, compelling use case.

Re: Gamification is not entirely bullshit

#27
post #19

Earlier quoted context omitted.

Seeing as you seem to be well-read in the area - what's your opinion of the "gamification" on khanacademy?

The reason I've been reading it all is because I'm writing my thesis on it :) I haven't spent too much time with Khan Academy, so can't say too much about it. There are two things to keep in mind: 1. Extrinsic motivation erodes intrinsic motivation. The most obvious example is asking a kid to take out the trash. Once you pay him/her, they'll only do it for money again. While this example is silly because no-one has a…

For me, Stack Overflow's most rewarding feedback is social recognition for having been helpful, which is largely independent of the formal rewards it offers. I bet it's the same for a lot of other people too.

Re: Gamification is not entirely bullshit

#28
post #19

Earlier quoted context omitted.

Seeing as you seem to be well-read in the area - what's your opinion of the "gamification" on khanacademy?

The reason I've been reading it all is because I'm writing my thesis on it :) I haven't spent too much time with Khan Academy, so can't say too much about it. There are two things to keep in mind: 1. Extrinsic motivation erodes intrinsic motivation. The most obvious example is asking a kid to take out the trash. Once you pay him/her, they'll only do it for money again. While this example is silly because no-one has a…

Loving the thoughtfulness here, and I have a few more resources I need to read now.

Re: Quick and Correct badges -

It is important in many cases to be able to recall math facts quickly. Automatic recall has also been correlated with understanding of more difficult concepts (for example: [1]). I am by no means saying that we've gotten the reward system perfectly right, but we were thoughtful when creating it.

[1] http://www.ldonline.org/spearswerling/Developing_Automatic_R...

Re: Gamification is not entirely bullshit

#29
post #19

Gamification is bullshit by design. Gameful mechanics may not be, but we're not there yet. Point 1 is that Zichermann (who wrote Games-Based Marketing) misrepresents the research into motivation. Whether this is maliciousness or incompetence is up for debate (personally I think the latter with a dose of arrogance). You can see Deturding shred his O'Reilly book, where he goes into this in a lot of detail [1]. One poin…

Seeing as you seem to be well-read in the area - what's your opinion of the "gamification" on khanacademy?

Based on going through nearly the entire Khan math test graph, I can say that the thing that was great was the instant feedback that the problems give you.

For those not familiar, you need to get many questions right in a row to move. These problems are generated along with "hints" specifically tailored to the problem - as-in the problem is worked step-by-step for you as you click "hint". What was also great is that it adapts (it seemed at least) to give you problems like the those you're having trouble on.

All of this gives you the feeling: "I'm really getting this, I'm not so bad at math after all" - all intrinsic. I looked at the badges as just "neat" - but the big chart of activity was somewhat inspiring because you can see yourself persevering. Then again - this is also intrinsic.

Re: Gamification is not entirely bullshit

#30
post #28

Earlier quoted context omitted.

The reason I've been reading it all is because I'm writing my thesis on it :) I haven't spent too much time with Khan Academy, so can't say too much about it. There are two things to keep in mind: 1. Extrinsic motivation erodes intrinsic motivation. The most obvious example is asking a kid to take out the trash. Once you pay him/her, they'll only do it for money again. While this example is silly because no-one has a…

Loving the thoughtfulness here, and I have a few more resources I need to read now. Re: Quick and Correct badges - It is important in many cases to be able to recall math facts quickly. Automatic recall has also been correlated with understanding of more difficult concepts (for example: [1]). I am by no means saying that we've gotten the reward system perfectly right, but we were thoughtful when creating it. [1] http…

I certainly did not mean to indicate I didn't think you guys had not put a lot of thought into things. If anything, my point was that all of these things are really hard, and I certainly don't count myself among the handful of people in the world who I would trust to get it right. I also have zero pedagogical chops, so don't take my word for anything!

My gut says that, if asked, I would probably recommend that the automatic recall goal is better placed as part of individual, fun "quickfire" lessons "Get these five right before the timer runs out". This would help to ensure you limit the behavior to a place where you want it, rather than somewhere where you might prefer the students to slow down and absorb what they're doing.

You could offer an informational badge showing the student completed a quickfire lesson, and I think you'd then be closer to kosher on my read of the psych literature.

But like I said, don't take my word for it :)

Post reply on HN