1

Setting the Stage – Introduction to QA for Modern Games

At its core, quality assurance (QA) in game development isn’t much different from QA in other types of software. However, there are some QA testing aspects that are specific to games.

But, let’s first start by introducing how QA is done in modern games, when, and by whom? How is it organized in this extremely fast-paced industry? These are some of the questions we are going to answer in this chapter.

In this chapter, we will first discover the main differences between the testing of games and the testing of other types of software. Then, the reader will learn more about the importance of QA. Finally, we will go through a couple of real-world scenarios that showcase what can happen when testing goes wrong in the gaming industry.

By the end of this chapter, you will have good insights into the basics of game QA and its importance.

In this chapter, we will cover the following topics:

• Understanding the evolution of modern game testing
• Exploring the differences between software and game testing
• Why is QA important for games, especially within the agile process?
• When and how should QA testing for games be performed?

Understanding the evolution of modern game testing

Today, QA is one of the key components of any modern software development process. It is unimaginable to release software to users without testing it first. Users now have so many choices with regard to apps, games, and digital tools, and if you release software that does not work properly or has usability issues, you risk losing many of your users. Even worse, you risk your reputation as a developer if players discover something in your game that doesn’t work and publish this information online on game forums and social media.

Therefore, QA is an important component of the development process.

QA and testing are interchangeable terms. Throughout history, humans have striven to provide quality of execution in their work – from the ancient pyramids through medieval fortresses to modern software. At its core, modern QA has its roots in medieval professional guilds, such as those for tailors, merchants, and smiths. To ensure that the quality of their products met the required quality standards, guilds implemented strict peer control that in many ways is similar to testing today. They set quality standards that guild members had to meet in order to become part of and stay in the guild. These parameters ensured that guild members everywhere provided a high level of service and in return, drove more business to them.

Modern testing is not too far from that: we test software to ensure that it meets the required quality standards and includes all implemented and approved features. Of course, these days, we have replaced the quality standards set by guilds with ones set by product owners and end users.

Modern game testing has developed along with the growth of the gaming industry. Games became widely popular in the 1980s, and they kept on evolving to various new platforms: first consoles, then home PCs, and, in the 21st century, mobile and other handheld devices. It doesn’t look like the industry is going to slow down anytime soon either. The gaming industry in 2021 was globally worth more than 180 billion USD, more than the music, TV, and film industries put together (https://www.thc-pod.com/episode/the-gaming-industry-is-now-bigger-than-movies-and-music-combined). The biggest money makers are mobile and free-to-play (F2P) games, and the top earners among them bring in over a billion USD per year (https://newzoo.com/key-numbers; https://www.statista.com/statistics/263988/top-grossing-mobile-ios-gaming-apps-ranked-by-daily-revenue/).

Games are a big business today and quality is more important than ever. With that said, there are no common standards for game QA across the industry. Every gaming studio is different and even within the same company, different teams follow different QA practices. The differences are even more significant when testing for different gaming platforms – while testing on consoles hasn’t changed too much since the beginning of game development, mobile game testing is embracing the latest trends in QA to be able to support more fast-paced development.

The terminology used in game testing is not unified (a particular term may mean completely different things in different studios), and even QA jobs might have different levels of responsibilities or completely different job descriptions from studio to studio. Taking into account all those differences, certain things remain the same. Testers generally spend months testing games repeatedly, using different approaches and shifting focus to different parts and characteristics of the game.

What would we consider to be QA today? There are many different definitions out there, but at its core, game QA is a set of testing activities, including test execution, exploration, and verification, that aims to ensure that games meet design specifications, technical quality, platform regulations, and player expectations. As we can see just from the definition, QA in gaming entails a lot of responsibilities and requires a whole range of skills. A game tester is a person who must fully understand the product’s vision, is familiar with technical risks and dependencies, can juggle conflicting priorities, knows the game better than anybody, and represents the players’ interests.

Now that you have a basic understanding of the prerequisites and best practices of QA for games, let us try and understand what makes game QA different from normal software testing. Knowledge of these differences, especially in terms of the unique aspects we test for in game development, is essential for us to master QA testing and meet the end user’s expectations within the gaming industry.

Exploring the differences between software and game testing

Regardless of whether our software is used in medical devices, spaceships, or the games we play, the testing methodology, if not the same, is remarkably similar. But, even if testing medical software might seem more demanding, it doesn’t mean that testing games is going to be any easier. It’s challenging to adequately perform QA on time and within budget. Usually, if development encounters problems and misses milestones, QA testing, which generally occurs toward the end of the development cycle, will also be postponed. Unfortunately, software release dates are rarely moved forward. Hence, the more delayed development is, the shorter the time span allowed for QA checks. However, the scope for testing stays the same or even grows wider. That’s why the profession of game testing is somewhat notorious for its long working hours and high stress levels.

As per their definition, games also comprise software, just like the app you use on your phone to track your steps, the software that assists pilots to fly planes more efficiently, and even the software you use to read this book! All these digital products have their differences – in terms of complexity, the programming language they use, the target audience, user experience, and so on. However, they also have lots of similarities.

Software that is used in airplanes, medical devices, and military applications is considered life-critical. That means that if such software fails for any reason, it can lead to the loss of human life. When we compare this to game software, the worst consequences of bugs in game systems are a loss of progress or, in the case of F2P games, a loss of money. While unpleasant for users, failures in game software are much less impactful.

Looking at the difference through the lens of QA, life-critical software testing is usually more rigid, takes longer, and has strict, well-defined requirements. When we talk about non-life-critical software, testing practices will very much depend on the internal company processes and software development methodology in use. Although some aspects of software testing are company- or industry-specific, the following aspects are common:

• Stability
• Scalability
• Functionality of features
• User interface (UI) look and function
• User flow and usability (UX)
• Performance under stress
• First-time user experience (FTUE)
• Localization

When testing games, in addition to the aforementioned aspects, we also test for certain specific aspects that are generally not tested in any other category of software development. These include the following:

• Fun factor
• Artificial intelligence (AI) in games
• Game physics
• Evaluation of...