Small Chance, Big Problem

"Black Swan" events in Game Design

I have mentioned the new game that David Thompson and I are working on - Zheng He - in a few GameTeks before this.

But to recap, it allows you to step into the shoes of Zheng He, who in the early 1400's led a huge fleet of 300+ ships and 30,000+ people on seven voyages through the Indian Ocean, reaching as far as the east coast of Africa.

I'm very excited about this design. As a solo-only campaign game, it is unlike anything I've ever done before, which has kept the design process fresh and engaging. Plus working with David (Undaunted Normandy, War Chest, Pavlov's House, and way more) has been a delight.

I think we're getting close to finalizing the design. We are mainly in the 'tweaking' phase, as we adjust card values, victory targets, and other parameters. However we're running into an interesting design issue that I wanted to share.

The Action System

First a bit of background on the Action system. Each turn you can perform one or two actions, and each turn there is a target for how challenging actions will be. If you want to do an action successfully, you need to play cards that match or exceed the target.

For example, at the start of the turn you flip up a card and it says you will need a '3' for success that turn. If you're doing a Resupply action you could play a Resupply 2 and a Resupply 1 from your hand.

It's a tad more complex than this, but you get the idea.

However sometimes the Challenge number is modified by a die roll. For example, you can decide to do two actions in a turn instead of one, and then for each the the Challenge number is increased by a die roll. When you have to roll a die, you decide what cards you're going to play before you roll. So you need to do some risk management.

For this mechanism we're using a custom six-sided die. The sides are:

0, 0, 0, 1, 1+, 2+

So 50% of the time there's no change to the target.

The 1+ and 2+ sides are exploding. This means that if you roll one of those, you make another die roll and add that to the first. If that also explodes, you roll again, and keep rolling until you get one of the non-exploding sides.

We chose the exploding dice mechanism because on average it doesn't affect the challenge number that much - the average result is about a net of +1 - but there is no value of cards that will guarantee success. Most of the time you're fine, but occasionally things can go spectacularly wrong. But we think that captures the feel of a sea voyage in this time, where things occasionally did go very, very wrong.

Most of the time, failing a Challenge means you just don't do the action, or do a less effective version. The Resupply action, for example, let's you draw more cards. The more you fail by, the fewer cards you get to draw.

But the Sail action - where you move around the board - is a little different. If you fail the Sail action you still get to move to your destination - not moving was too big a hit to planning - but you must add damage cards to your deck (a number of damage cards equal to the amount you failed by). If you played 5 points of worth of Sail, but the Challenge number ends up being 8, you add 3 Damage cards to your deck. During a voyage you can take a Repair action to remove them from your deck.

Standard deck-building stuff.

The Problem

This is a board game, not a video game. So the number of Damage cards we can include in the game is inherently limited. Given the scale of the number of cards that are included, we're thinking twelve damage cards seems about right.</p><p>Next obvious question: What happens when the damage cards run out and you need to add more to your deck?

It needs to do something bad - otherwise when you reach the max you can just act with impunity. There's no more penalty for missing the challenge number.

The natural rule, and the one we're currently using, is - your fleet sinks, and the game ends. Thematically fits, and is simple.

Except that the exploding die gives us a potential issue. It is possible that someone could match the 'nominal' challenge number with their cards, but just through sheer bad luck completely sink their fleet by getting all 12 damage cards in one shot, from repeatedly rolling the 1+ or 2+ numbers.

But what's the actual chance of that happening? Data always makes these decisions simpler, so I fired up my dice analysis tool, McDie (freely available over on itch.io), and built this simple network:

And found out that the chance of rolling a total of 12 or higher on a single die is about one in 4,500. The chances of a 15 or higher are about one in 20,000.

Small Numbers, Big Problems

Let's say I'm a new player to Zheng He. I just broke the shrink, set everything up, sit down to play, and confidently sail out of China. Then I roll and get a 2+, then a 1+, then another 1+, then a 2+, and.... after a few more rolls my fleet sinks and the game is over.

That's a terrible player experience. I went from a fully healthy fleet to game over in a single action.

Dealing with it will require adding more rules. There are many ways of adjusting this, but all will require more rules overhead. And wherever possible we try to eliminate rules. Every rule should have a purpose.

So... is it so bad to have a terrible player experience one out of 4,500 games? No one is going to play a game this many times, right? And if we increase to 15 damage cards, it's only one out of 50,000 games. That is incredibly, incredibly rare. Maybe we don't need to address this at all.

Well, yeah. But let's look at it another way.

Zheng He is going to be a niche game - let's face it. It's not the next Exploding Kittens. So let's say it will sell about 3,000 copies, which is typical for a hobby game.

Each voyage will probably require a die roll on a Sail action about 10 times. And there are about 7 voyages per campaign game.

That means there will be: 3,000 x 10 x 7 die rolls, which is 210,000 die rolls. More if people play the game more than once.

Now 1-in-4500 or 1-in-20,000 isn't looking so rare. It's actually looking like a certainty that it will happen. In fact, if we keep 12 Damage cards it will probably happen 50 times. 4 times if we go to 15 Damage cards.

Maybe you think you can live with 50 incredibly unhappy players. Again - we're talking about a game-ending series of unlucky die rolls with no chance for remediation. But we don't want to take that chance.

You Know We're Living in a Society!

In the end we decided to add an additional option for players on the Sail action: TURN BACK. After you see how many damage cards you get, you can decide to cancel the Action and remain in your original port. If you do, you only take half the damage cards.

Normally I don’t like to add extra rules to deal with issues if possible. However, in this case we felt that it was simple, thematic, and gave players another decision point. Psychologically it also makes players more accepting of taking damage, because they do have an option to mitigate it.

The astute among you may have realized that this still doesn’t eliminate the problem. If you get 25+ damage, even if you take half you still sink the fleet.

However, the chances are now dramatically reduced - around 30,000,000 to 1. Given the audience of the game, this should theoretically never happen. And if it does, well the player has gotten a heck of a story out of it.

I've seen this issue with many game designs that I've helped playtest or evaluate. There's an incredibly unlikely card deal, or combo, or something similar that will break the game. Way too often the designer will hand-wave off the issue, saying "it's a one in a million chance - players will never see it".

Be very careful when making these judgements. First off, it probably is more likely than you realize. Do some math and figure out exactly how likely. Second, evaluate how much of an impact it will have, and how bounded the effect will be. If it's a small effect, don't worry about it as much. If there's a 1 in 5,000 chance a player will want to light your game on fire, there's a 100% chance you should address it. Think about all the plays of your game - not just one.