How to Create a Mobile Testing Strategy in Three Steps

It used to be that when you left your apartment, you checked to make sure your keys and wallet were with you. Today, your smartphone is priority number one. So much of our daily routine is influenced by a smartphone that we literally feel handicapped without it.

According to GSMA Intelligence, more than 5 billion people worldwide currently use smartphones and mobile phones; that is two-thirds of the global population. In total, those 5 billion people making more than 9 billion mobile connections worldwide. No other device in the history of mankind has experienced such rapid market penetration. Entire areas of life are now influenced by the devices, and applications are now being (visibly and rather inconspicuously) integrated into cars, wearable technology, home appliances, and more to come.

Because of mobile’s omnipresence and the fact that many daily situations rely on flawless app experiences, mobile users have incredibly high expectations when it comes to digital quality. Companies realize this and are beginning to prioritize mobile before all else. In the following, I have devised a three-step guide on how to create your own mobile testing strategy.

Mobile Testing Challenges to Solve

One of the biggest challenges a team must solve for is device fragmentation. There are more than 25,000 Android devices alone on the market. Add that to the smaller, yet still complicated iOS device market and you have a difficult fragmentation issue on hand. Further complicating the situation is user mobility. This takes into account several variables when using the app like location, conditions, and network preference, to name a few. And depending on the target customers and the target country the mobile platform usage differs as well.

Mobile devices are only the tip of the iceberg, too. Wearables, fitness bands, IoT devices, and e-mobility services continue to flood the market, adding a new layer of mobile connectivity. Therefore all these devices should be considered when creating a mobile testing strategy.

A Solid Mobile Testing Strategy in Three Steps

Step #1: Identify Your Target Customer

With this data, a team can select the most appropriate mobile devices and wearables from their customer base and group them into priority clusters. This approach will help to reduce the number of test coverage during the development and testing phase. From here, the big decision is whether you should buy all the test devices, use a beta testing community approach, or try a crowdtesting solution.

A team has to decide for one of the options or a mixture of them. Important is, that testing on simulators and emulators is nothing that should be considered as the only test environment. Testing on simulators and emulators is perfect in an early development state, but when it comes to testing in-the-wild and in the usage scenario of the app, testing on real devices is a must.

Step #2: Define and Gather Your App Requirements

With the help of a designer (rather an interaction designer) and real users, app requirements are needed to define possible test scenarios. Based on these scenarios, your mobile development team can decide which devices will be used, which test data is needed to simulate the user behavior, as well as which environment the app will be tested in.

Here are some sample scenarios:

• The user must be able to log in to the app.
• The user must be able to register.
• The user must be able to search for products.
• The user must be able to view the shopping cart.
• The user must have the option to choose between different payment providers.
• The user must be able to synchronize the mobile app with wearable devices.

Scenarios like this will help to focus on the critical parts of the app and to prioritize the work during the development phase.

Step #3: Define Your Mobile Testing Approach

It is a process to create a mobile testing strategy, but if you can answer the following four questions, you’ll be well on your way down the right path.

1. Which part of the app can be automated and on which layer?
2. What are the parts that can’t be automated and need intensive manual testing?
3. Are there any hardware specific functions used by the app?
4. What are the non-functional requirements of the app?

Mobile test automation plays an important role in the whole software development lifecycle of an app. However, it is not an easy task and should not be underestimated. When thinking about mobile test automation, keep the test automation pyramid in mind and decide on which layer a feature should be automated.

With the mobility of users, not every feature can be automated in an app. Depending on the use case of the app, automation on the end-to-end layer may not be possible at all. Therefore, it is important to identify the features which are difficult or not possible to automate. These features must be tested manually before every release.

While performing manual mobile testing, it is important to test the app in the environment where the customers will use it. Possible test environments could be your daily commute on the train, a trip into the woods, or while participating in a sports competition. As mentioned earlier, crowd or beta testing might help in this phase.

If the app uses hardware-specific functions like the camera or the different sensors, these functions must be part of the test strategy and must be prioritized based on the user scenarios. When testing against hardware-specific functions, it’s important to test on a wider range of test devices to cover the different hardware components.

Last but not least, the mobile development team must know the non-functional requirements of the app.

Non-functional requirements to think about are:

• Security
• Load & Performance
• Usability
• Accessibility
• Internationalization (I18N) and Localization (L10N)

It is important to plan these non-functional requirements into the design, development, and testing phases.

Prevention is better than a cure

As the software moves forward in the SDLC, the cost to fix bugs increases exponentially. With the help of a comprehensive mobile testing strategy, bugs can be identified in earlier stages while keeping the overall process fast, lean, and efficient.

Stop compromising on digital quality and take the time to choose the proper testing approach for you. Your decisions now will make all the difference down the line.