Test your damn game: choosing participants
It is important to test your game with representative users. Talk to them, watch them play your game. You need their independent feedback, after all you are too close to your game.
Choosing the right participants is key to success
It is important to get the right participants for your user tests. You will be using the test feedback to change or improve your game. You should try to make sure the participants are the right audience for your game so you don’t waste time or money.
Would you trust anyones feedback on your game? It is important to match the feedback with the type of person who gave the feedback. For example: Who said the combat is boring? Is it someone who isn’t a fan of your game’s genre? Is it a super fan who has spent $1000 in a competitor’s title?
You might think that anyone is appropriate for UX or FTE testing because you expect the game will be intuitive enough for everyone. For most tests you want a specific audience.
Principles to identify your audience and your user testing participants
Quick! What is the target audience for your game? If you answered, “Males 15-25,” then you lose!
A demographic is not your audience. If this is how you select your participants then you are unlikely to get useful feedback. Simple demographic information is insufficient to know if a person will be a fan of your game. Defining and categorizing participants is key to getting trusted feedback from user testing.
A person’s interests and past behavior is a much better indicator if they will like your game. So, we can define an audience through two major factors: 1. interest, 2. behavior. An interest means the person likes this type of game (genre, theme, platform, etc). Behavior means their experience or level of familiarity with this type of game.
4 player categories based on their behavior
Here are a few ways of categorizing players based on their behavior and what types of user tests are appropriate for each category.
1) Average User
Average users are the majority of the users. They have experience playing several similar games, or at least one title deeply.
Even though they are called average, they are not one homogeneous group.
Games are generally designed for this audience.
Average users are good for most types of users testing.
Expect participants from this group to require extra prompting to give feedback. They might not want to say anything negative.
2) Power User
Power users are the hard core super fans of this type of game. They have played at least one similar game deeply and competitively. They have probably spent money in a F2P game.
Power users are focused on the fastest route to power in your game. They want to constantly play and advance.
Power users are best used for tests focusing on meta game, progression, competition, or monetization.
3) Novice User
The novice user does not have much experience playing this type of game. They may lack an understanding of genre specific conventions or mechanics. Examples: energy systems, random boxes, turn based combat, etc.
Since they don’t have the same base level of experience as an average user, they will experience issues that other players won’t.
Novice users are best used for First Time Experience tests or usability tests.
Expect confusion that isn't representative of main audience when testing with novice users.
4) Influencer, Maven
Influencers are early adopters who play lots of games, stay up to date on gaming news, and talk to lots of people.
They understand what other influencers like and hate. So, they can prevent you from making a big mistake with your audience.
Influencers are best used to give you general feedback about your game. Keep in mind they do not speak for the average user.
3 practical tips for creating participant requirements
Let’s review some practical tips for screening your participants based on their interests and behaviors.
1) Interests - focus on comparable titles
Focus on what comparable titles the participant as played. A comparable title is a game like yours that participants might have played. When someone has played a comparable title, it is a strong indicator that they like that type of game.
What makes a good comparable title?
Shares some main gameplay concepts or features from your game
Optionally, has similar theme or art style to your game
Is large enough so you can find people who have played it
2) Player Behavior - what did they do in the comp title?
When screening for player behavior you should focus on what their play-habits were with the comparable title. You may want to know how far they advanced, how much money they spent, how often they played, how competitive or social they were.
What did they do in the comparable title?
How far did they advance: e.g. what level did they reach
How much real money have they spent (for a F2P title)
3) Don’t be too restrictive
Avoid being too restrictive with your requirements or you won’t have enough participants. Avoid requiring having played too many comparable titles.
Participant screening process
Before we run a user test with potential participants we need to determine their suitability for our test. We do this by asking them questions to see if they meet our requirements.
