In last week’s GameTek I described a thought experiment about retheming Incan Gold as a firefighting game. A few years after I first published that, I was contacted by Dr. Stephen Blessing at the University of Tampa, asking for my permission to actually conduct the study! I was thrilled of course and am pleased to present those results here.
I have to agree with Matt. It appears you may be over-generalizing from the data. Definitely by Game 4, there are no differences between time spent playing the games. I think one big takeaway would be Theme hardly matters. (by game 4, theme doesn't seem to be impacting time spent processing - however, it might actually still be hindering performance).
In Table 1, the "rules are difficult to understand" is insignificant across all conditions - at least with a cell size of 100. Yes, there is a significant difference in Table 2 between abstract and adventures, but I wouldn't find this one difference a compelling finding - after all, there are 33 pairwise comparisons - I'm not surprised that one is significant...
I think it is an interesting study. One question that intrigues me is since each game was fixed, we know what the optimal series of choices was for each game (either based upon card outcome or by expected values updated after each card is revealed). When participants performed sub-optimally in each condition, we could check if it was because whether they took too much or two little risk in their decisions to stay or return to base camp/leave the building. An interesting analysis would be to see if players acted more "riskily" in one condition over another.
@geoff it's nice to see any data on these topics, and this is definitely a fascinating one.
Forgive me if my statistics are rusty, but +/- 1 SE shows that the true population value for these measures has a ~68.2% chance of falling within the error margins right? Do you know what these look like at a 95% confidence interval?
It would be interesting to look at the variance of the scores. did the "firefighters" took more risks and more of them finished the game with 0 points, lowering tha average? Or did they play more safe and rescued fewer "points", trying to at lease save some people from the fire?
I see a difference between "If you push your luck too much you die, and loose the treasure you collected so far" and "you die, and so do all the people you rescued so far".
Hi all! This is Stephen Blessing, the person who did the experiment. Special shout-out to my undergrad assistant who tested most of the participants, Elena Sarkosky. She also did the additional art we created for the variants and the backdrop (I didn't know when she came on board she had some artistic talents!).
If you have questions, feel free to post here obviously, or send me an email at sblessing at ut.edu. A pre-print of the paper can also be found here: https://doi.org/10.31234/osf.io/syjm9 (this takes you to an ArXiv site; this version only has the first experiment). This will find a journal home someday, but it kind of falls between the cracks of topics that cog psych journals find interesting (where I mostly publish).
I have to agree with Matt. It appears you may be over-generalizing from the data. Definitely by Game 4, there are no differences between time spent playing the games. I think one big takeaway would be Theme hardly matters. (by game 4, theme doesn't seem to be impacting time spent processing - however, it might actually still be hindering performance).
In Table 1, the "rules are difficult to understand" is insignificant across all conditions - at least with a cell size of 100. Yes, there is a significant difference in Table 2 between abstract and adventures, but I wouldn't find this one difference a compelling finding - after all, there are 33 pairwise comparisons - I'm not surprised that one is significant...
I think it is an interesting study. One question that intrigues me is since each game was fixed, we know what the optimal series of choices was for each game (either based upon card outcome or by expected values updated after each card is revealed). When participants performed sub-optimally in each condition, we could check if it was because whether they took too much or two little risk in their decisions to stay or return to base camp/leave the building. An interesting analysis would be to see if players acted more "riskily" in one condition over another.
@geoff it's nice to see any data on these topics, and this is definitely a fascinating one.
Forgive me if my statistics are rusty, but +/- 1 SE shows that the true population value for these measures has a ~68.2% chance of falling within the error margins right? Do you know what these look like at a 95% confidence interval?
It would be interesting to look at the variance of the scores. did the "firefighters" took more risks and more of them finished the game with 0 points, lowering tha average? Or did they play more safe and rescued fewer "points", trying to at lease save some people from the fire?
I see a difference between "If you push your luck too much you die, and loose the treasure you collected so far" and "you die, and so do all the people you rescued so far".
Hi all! This is Stephen Blessing, the person who did the experiment. Special shout-out to my undergrad assistant who tested most of the participants, Elena Sarkosky. She also did the additional art we created for the variants and the backdrop (I didn't know when she came on board she had some artistic talents!).
If you have questions, feel free to post here obviously, or send me an email at sblessing at ut.edu. A pre-print of the paper can also be found here: https://doi.org/10.31234/osf.io/syjm9 (this takes you to an ArXiv site; this version only has the first experiment). This will find a journal home someday, but it kind of falls between the cracks of topics that cog psych journals find interesting (where I mostly publish).
Thanks for everyone's interest!
My takeaway - themes matter less than most people think, and we should make more abstract games. :)