3 Reasons why Remote Testing is Critical to Ensure Quality Digital Products

The switch to remote work during the coronavirus crisis is leading many companies to take a second look at their work habits and processes. As highlighted by Constantin Büker in his recent article, many QA, product and dev teams must rethink their processes as they have now moved to a fully remote setup.

In this context, leveraging remote testers is an efficient way for QA teams to address the challenges that have arisen with the coronavirus crisis, specifically the impossibility to test under normal conditions. But it isn’t just that.

By utilizing remote testing practices on a regular basis under normal circumstances, QA, product and dev teams gain efficiency while ensuring they build a truly customer-centric product.

Here are three instances where it makes the most sense to leverage remote testing:

1. Localization: In-market insights are essential

Let’s start with the most obvious one. Whether you’re launching your app in new markets, or simply want to ensure your newest feature functions properly and provides a good experience to your users in different markets and in all languages, you cannot really localize your digital products unless you’ve tested them with local users whose demographic profiles, cultural backgrounds and preferences match those of your customers.

Let’s use the example of voice testing and AI training. In order to properly train the AI engine to understand just a couple specific commands in one language, QA teams must collect data from a large group of testers who are sourced from different age groups and genders, and who speak with various accents. Each of these testers will need to execute many unique voice utterances in various contexts.

In other words, in the case of AI training and voice testing, collecting a diversified set of data is absolutely essential in order to avoid bias. Moreover, in other situations such as content localization or product launches into new markets, utilizing in-market insights from real users with a very precise profile and background is crucial to ensure user centricity. As you can imagine, achieving this is impossible without leveraging a remote, global community of testers.

2. Geographical coverage: You can’t “fake it until you make it”

No matter how advanced VPN technologies may be, using a VPN to test a digital product in various “locations” inevitably causes unexpected issues. QA teams relying solely on these tools will always face some uncertainty as they can’t predict how they affect the functionality and interface of their app.

According to Sarah Franco, Testing Services Manager at Applause, “solely relying on a VPN to simulate testing in various locations can be tempting. However, it is definitely not a viable option.” Using the example of testing geo-localized ads for streaming platforms, she explains that most streaming services automatically detect the usage of any VPN tool. “In some cases, using a VPN can even alter the test results. For example, by causing a delay in loading the ads we want to test. Obviously, this creates biased and unreliable data, which we cannot work with.”

The unwanted consequence of using a VPN is just one example among many. Other examples include:

• Using mobile data on specific networks in the world
• Switching between mobile data and a Wi-Fi connection
• Testing GPS-related features
• Payment testing on specific mobile networks

Many of these setups create a real challenge and a variety of possible outcomes that cannot be reproduced by in-house testing.

Solely relying on a VPN to simulate testing in various locations can be tempting. However, it is definitely not a viable option [...] as it can alter the test results.

Sarah Franco, Testing Services Manager EU, Applause

3. Device coverage: Have you considered all edge cases?

In today’s extremely fragmented device landscape, QA teams face increasing difficulties in keeping up with the myriad device/OS combinations they need to test on. For obvious reasons, purchasing and maintaining an expansive pool of devices is not an option; not only is it too expensive, it’s time consuming and harms the team’s productivity.

Beyond the logistical aspect, QA teams must collect data from real-world testing to make informed decisions. No matter how many test cycles are run, QA teams can’t uncover all unexpected bugs and edge cases unless they include in-the-wild testing in their strategy.

“In addition to testing on a large amount of device/OS combinations, you also have to think about the situations that can only be reproduced in real life,” Sarah Franco explains. “When it comes to hardware, a very good example would be when QA teams test pairing some hardware with a given combination of ISP/router. You have to test it in real conditions — there is no way around it.”

In order to test anything related to hardware in an efficient manner, leveraging a community of remote testers is the best way to obtain reliable results without burdening internal QA resources, generating disproportionate costs or missing out on important use cases.

If you're considering switching to a remote testing setup, read more about what to expect when switching to a fully remote QA and dev process or find out more about remote testing on our resources page.