Like peanut butter and chocolate, testing and community is a very tasty combination. In years past, testing and community typically operated at different sides of the production cycle spectrum. But more companies are recognizing how a skilled community engagement team, working hand in hand with the testing team and starting much earlier in the process, can ensure a seamless customer experience and build the foundation for your post-launch community while capturing vital community feedback during the beta phase.
In this case we’ll be examining a free-to-play (F2P) game scenario. A F2P game is a video game in which players can download and have access to almost all content in the game without paying any money. F2P games come in many different shapes and sizes, but the most common are those based on the freemium model. That is, players can enjoy a fully functional game for no charge at all, but if they want any extra bells and whistles (flashy avatar gear, specialized guns and so on) they must make micro-transactions to obtain them. The opposite of a F2P is a pay-to-play (P2P) game in which payment is required up front to play the game. F2P games in particular have many intricacies when it comes to functionality and revenue generation, and therefore there is much to learn from real, individual users during beta. Gathering feedback about how people actually interact with the game and what triggers might make community members make the leap and actually purchase in-game items is absolutely golden information to have while you are operating in a phase where major changes to game design are still possible.
During the planning phase, several aspects need to be laid out for examination. The purchasing process via the in-game store needs to be intuitive and quick for users. Also, F2P games can run the risk of putting players who choose not to pay at too much of a disadvantage, causing them to leave before they are committed to the game enough that they might eventually choose to make purchases. Also, the number of store items and item packs need to be extensive enough to keep people interested in coming back for more — you don’t want players to burn through everything too quickly! During the planning phase, the test team should keep track of all the various general F2P considerations and those unique to this particular game, and begin development of the documents that will be used to bring the community moderation team on board.
Testing for F2P games typically progresses through these four stages: in-lab testing, closed beta for family and friends, closed beta for the public and open beta.
Onboarding of the community moderation team should start during the closed beta for family and friends phase. At that time the moderators should be given a crash course by your test team on everything related to the game. Specifically for F2P games, the mods should be educated about the habits and needs of the following player types in order to prepare for collecting their feedback and interacting with them within the community (Figure 1):
Roadrunner: a player who wants to play through the game as quickly as possible.
Socialite: this person plays with others using the friend and clan systems. He or she changes servers and modes frequently and communicates a lot.
Role Player: someone who focuses mostly on the game’s story. This is someone who listens carefully to voiceovers and reads the descriptions of everything.
Gold Digger: this player is hellbent on hoarding cash, and just looks for ways to get money as fast as possible.
Moneybags: someone who spends a lot, buys a lot and places importance on having the newest, shiniest gear.
Loser: a player who loses all the time, is always last and drags the rounds for other players.
Explorer: someone who focuses mainly on map exploration and trying out all the items. This person attempts to access difficult locations in the maps and may also try out all of the various inventory items available (weapons, clothing, add-ons) and all of the different combinations of those items.
Destroyer: this person enjoys trying to break or exploit every single aspect of the game in order to find the harder-to-discover bugs.
By the time the production cycle reaches the closed public beta stage, the test knowledge should be completely transferred to the community moderation team. An internal game Wiki should be complete, as well as terminology lists and frequently asked questions. The moderation team needs to know what is challenging about this game, what the known issues are, and what aspects of the game the test team suspects will receive the most feedback.
Beyond documentation of game functionality and quirks, the moderation team needs to have been trained on how to categorize community feedback and sentiment level in a way that will be useful to a test team. Take a community comment, for example, which might come from a forum (Figure 2). This comment is referencing the fact that the player is stuck on the terms of service agreement screen and therefore can’t download the game.
Moderators will be gathering reams of information like this through community channels, and documenting it in a way that makes sense to the test team will turn this community feedback into something beneficial, rather than it just being noise coming from the community team in a random email. A system of reporting bugs that incorporates correct terminology and assigns levels of severity and priority in line with how the test team is used to working will ensure that all these snippets of community conversation will directly improve the game. The moderation team needs to be trained on how to do this, and how to work with testers to make sure they will not be wasting any time.
Open beta begins
By the time you hit the open beta phase, you will have assembled a crack team of moderators and a community manager to manage them. In order to work effectively with the test team as well as perform the necessary tasks for their community engagement responsibilities, it’s important that your team have the appropriate skillset for the job.
All of the skills that make for a good moderator are also helpful when it comes to getting good information back to the test team. First and foremost, the moderators must all be bilingual, in English plus whichever target foreign markets are important to your client. This will not only help them serve the community better when it comes to engagement post-launch, but it also ensures that they will be able to seek out, capture and respond to bugs in their native tongue as well as English during open beta. Additionally, they must be subject matter experts in the product. In the case of our F2P game scenario, this must be a team of gamers. Gamers will not only connect better with the community, but they will speak the language of games and be able to recognize and distill feedback quickly about bugs or design change requests due to the fact that they speak the language of games. Also important is that these folks have experience moderating forums and social media. This is of course crucial for when they are in full-on, community engagement mode. But during beta as well, experience with navigating social media channels will make gathering information for the test team that much easier, since the moderators will simply know where to look.
Open beta is where the fun begins! This is a period of intense collaboration between the test team and the community, and all of the groundwork you’ve laid educating the moderation team will bear fruit in the form of appropriate, real-time gameplay and user sentiment feedback about the brand that is actually actionable by the test team.
Also, this is a unique, often overlooked opportunity to establish a solid foundation for your post-launch community. By getting publicly involved during open beta, your moderation team will begin to establish camaraderie with the community members, and will achieve “street cred” that isn’t possible when you jump in after the painful process of working out all of the bugs is already over. According to Adam Keating, senior account manager of game services, “community members, particularly when it comes to games, need proof of expertise before they let their guard down with officials. Open beta is the perfect chance to establish that rapport, side by side in the trenches.” Once the beta phase is complete and the game launches, your team will already be known as helpful, knowledgeable representatives of your brand and will have forged special bonds with the most vocal and active super users in your community. These bonds will jumpstart the growth of your post-launch community.
Also, as the moderators collect feedback and begin engagement with the users, they will begin the process of purposely driving the community to official brand channels (Forums, Facebook, YouTube, Twitter, Vine, Twitch.tv). Depending on your regional community needs, they will either drive your community to official channels on regional social media channels such as Vkontakte, Werkenntwen and Renren, or they will find the regional communities that exist there and drive them back to your official English channels.
3-2-1, blastoff!
Once all the phases of beta are complete and your game has launched, your moderation team fully takes the wheel and switches to a program focused on engagement, entertainment and support. Now that you’ve released a game that has been refined through a true community effort, your moderation team can start taking the community away from a focus on fixes and toward a focus on fun. While tracking critical issues and design change requests will be ongoing throughout the life of the project and there will be ongoing testing updates to evaluate the community feedback, the focus now turns toward the end user experience. The team begins soliciting and highlighting stand-out community contributions (images, videos, forum contributions) and executing community campaigns focused on acquiring and retaining players and continuing the overall health of the community.
Everyone has the same end goal — create the best game possible in the eyes of your community. Your test team has the technical knowledge and experience to create a solid blueprint of the game. Your community team has the relationship with the players to determine what’s been overlooked and what fine tuning needs to be done. Put those two skillsets together by creating a framework for communication between the two teams, and you’ve got yourself a winning combination.
