You ask questions, I find answers.
Really… I want this topic to be that simple. My purpose for this blog has always been to share about my experiences in the game industry. Beyond just my experiences, what interests you about the game industry, freelancing, Gen Con, or other game-related topics? From your questions, I will find topics to research and then share my findings with you.
This second Fact-Finding has resulted from another question posted in The Tavern (the InnRoads Ministries‘ Facebook group) by Zach Lorton. He has a game design question which had me reaching out to publishers and designers I know.
Question from Zach Lorton:
When playtesting, is there there a minimum number of playtests that you like to shoot for? Assuming you account for changes to be made during the playtesting development phase, of course. And do you prefer to test with differing numbers of players?
From Lance Hill (game designer)
I think 100 playtests is a nice round number to shoot for. The playtests don’t need to be complete games, sometimes a person only needs a few rounds of play before they see something is broken and the game needs to stop. Playing with a different number of players is important, but how many times with the various player counts probably depends on the type of game that is being created. Something asymmetrical would require much more work at various player numbers than something that more easily scales up like, say, a drafting game or something on that line.
From Matt Leacock (game designer)
The more groups the better, but how you test is just as important than how many playtests you conduct. For example, you can get much more data by remotely observing a blind test than you can by simply asking a group to fill out a survey after they play.
I try to test with groups of different backgrounds, experience levels, and player counts. Early on, I play with expert players (who have less trouble dealing with cruder prototypes) and then branch out to more casual players as the product tightens up.
In order to get that kind of coverage, I’ll typically work with about a dozen groups. If I’m lucky enough to have a co-designer, we’ll can get even more data as we’ll have more bandwidth to create more prototypes, recruit testers, and observe more sessions.
Chris Birch of Modiphius
As many playtests as feasibly possible. The more you can do the better the game will be. Especially ensure that people you don’t know playtest the game without you being there to explain the rules. Recording ‘blind playtest’ sessions is a good idea so you see the mistakes people make in interpreting the rules or setting up the game. Analyzing the feedback is equally important and you should make sure that you establish the questions in a way that doesn’t skew the results.
From Stephen Buonocore of Stronghold Games
There is no ‘minimum’. It of course depends on many things including but not limited to how complex the game is. You definitely do want to play with all the player counts, again, depending upon that complexity. You cannot playtest enough. Above all, you have to playtest with BLIND-PLAYTEST groups. Your mother will tell you that it is the greatest achievement ever, and your friends might think you are so cool, but strangers will give you much much better feedback. Finally, all of this should be done by the designer BEFORE it gets to a publisher. A publisher should never have to take an “alpha stage” design through deep blind playtesting. Ensure the game is solid and ready for prime time before pitching it to anyone.
If I receive further responses to this question, I will post them in the comments below, so you might want to check back later on this post.