Digital quality isn’t a serendipitous result, rather, a constant commitment to building a product that a customer will love. Payment testing is a big component in that multi-faceted effort as brands peddle promotional offers and strive for global expansion.
With dozens of streaming media subscription options on the market, a commitment to digital quality and secure payments goes a long way toward the bottom line. It all starts with a very in-depth and iterative testing approach prior to launch, follows with a tactical and urgent strategy on launch day, and continues into the weeks and months ahead as in-market customers provide valuable feedback on early adoption. Neglect one step in the chain, and it all can fall apart, especially in payments, where one bit of friction in the many end-to-end touchpoints of a transaction can quickly result in a lost subscriber.
At Applause, we work with some of the largest media brands in the world, helping them succeed in securing their revenue streams for new-product or new-market launches. Let’s walk through some of the aspects that help these media brands succeed with payment testing.
What end-to-end payment testing looks like
To grasp the concept of effective payment testing, let’s first explain what payment testing is (and what it isn’t). Payment testing entails the following in a streaming media context:
confirm that new subscribers have the appropriate level of access to content during a free trial;
verify subscribers are charged (and merchants are collecting) the correct amount with each transaction across various payment instrument types;
validate successful refunds, transfers, and earning and redemption of rewards, if applicable;
ensure proper conversion of free-trial subscribers to the right level of paid subscription, including the correct charges — and vice versa, that free trial customers who choose not to subscribe are not charged;
help identify root causes of payment issues reported by subscribers or indicated by analytics;
verify subscription renewals/recurring charges.
These activities help eliminate friction in the customer journey of a media subscriber, which often results in cancellations, even more so than in other industries. However, other functional and usability issues can hinder a subscriber’s experience with a media platform — not all fall under the umbrella of payment testing. Such defects include:
bugs in the payment flow leading up to the payment step, but not actually attempting a payment transaction;
defects in the payment flow that allow the execution of a transaction with fictitious PI information (e.g. fake credit card numbers);
loading, adding or registering a payment instrument with an account or profile to enable certain features to test, but not actually attempting a payment transaction;
signing up for a subscription or free trial that doesn’t result in a charge to the tester.
As you can see, there’s a lot to test, and just as many different ways to test it. Many large organizations with global payment portals tend to focus their QA operations on payment acceptance and compliance, which can leave gaps in coverage when real subscribers try to pay.
New payment instruments hit the market every year, regulations change, and digital wallets and cryptocurrencies are fluid. This means payment testing isn’t a checkbox item on the way to release; it’s an ongoing commitment to ensuring strong revenue streams.
Pre-launch: Plan, prepare, push left
As with most software launches, it’s important to get the basics right. While any market can have a multitude of payment instruments, the goal is to achieve coverage for as many as reasonably possible. Start with the basics: support for credit and debit cards, as well as bank accounts. As part of our pre-launch testing with customers, Applause sources testers who have accounts with the specific banks our clients request, which is extremely important when it comes time to launch.
While some payment instruments are more popular than others, do the research to identify the ones most relevant to your business in each market, as trends can shift the balance. For example, Applause’s bug and test case data shows that credit and debit cards accounted for more than double the next most popular payment instrument method (bank accounts) in 2022, but alternate payment methods and e-wallets also increased. As consumers exercise their payment options in the marketplace — and as those industries consolidate and shift — media companies must cover the methods most in demand from subscribers.
Complicating matters more is that some companies will use different payment processors in different regions, or use a backup processor. In these instances, cross-check the data that you receive from the payment processor, and make sure you validate these payment flows as well. Just because an in-market payment instrument works for one processor doesn’t necessarily mean it will work for another.
Shift-left testing applies to payment testing as well, even if you can’t test the full end-to-end payment flow. At Applause, we often partner a tester with a dev engineer. When the dev is ready to process, the tester provides a real payment instrument to use and validate in real time. This shift left in payment testing gives a base level of confidence in the build’s ability to capture a payment.
The pre-launch development and testing process can take anywhere from weeks to months depending on the lifecycle. Make the most of your time and resources.
Launch: Test it before your customers do
When it comes time to launch, time is no longer on your side. While release strategies vary between organizations and industries, one common thread is to release the product in a controlled or limited manner to limit the exposure of any defects that escape to production. This is especially important in payment testing, where production is the only environment where you can sufficiently test the end-to-end payment flow.
On launch day, our customers typically release during the very early hours of the morning in that market. During this time, we hit the app hard, validating any number of potential usability, functional or payments issues.
Applause uses its million-strong community of digital experts to conduct testing in all relevant markets, on every supported platform, and with every supported payment instrument. In-market testers validate as many payment operations as possible — subscriptions, one-time purchases, refunds, etc. The result is a fury of testing that lasts anywhere up to a couple weeks post-launch. This diversity of testers and instruments helps identify any issues that require immediate fixing before the broader user base wakes up to use the product.
Sometimes, errors in the payment process are frustratingly simple. I’ve heard of one media company losing out on potentially millions of dollars of revenue simply because the payment processor forgot to click a checkbox to allow Visa payments in the new market. The sooner you discover a defect like that in production — the only place that particular defect can be discovered — the sooner you can take action to avoid disaster.
Post-launch: Don’t ease up
Many media companies cut back on payment testing once the product is live in-market — this is a mistake. As we’ve established, there is only so much that you can payment test pre-launch. That leaves quite a bit of validation to perform in the production environment, and that’s why Applause snaps into maintenance mode in the weeks following the launch.
Post-launch, Applause tests all payments within the regression test suite. Some organizations try to accomplish this testing with a VPN or test cards, but neither of these approaches account for local payment complexities and other issues that might occur in the payment flow. Organizations conduct rapid testing on issues found after launch. Eventually, testing slows into an ongoing maintenance mode to ensure payment issues don’t return.
Keep in mind that just because subscribers’ payments will process doesn’t mean issues won’t arise after launch. Streaming media bundles and promotional offers are common, and defects can occur along these e-commerce chains. Organizations must test these promotional combinations ad nauseum, ideally with users that fit the customer journey profile. For example, if a Visa customer is supposed to receive three months of a subscription service for free when they link their card, that requires ongoing testing. What happens after three months? What happens if they cancel their card? Are they served the appropriate messaging, and notified that the promotional period will expire?
The same goes for customer communications, which must be properly localized and relevant to the in-market audience. You can’t hold up a promotion because of your inability to test, especially when you already have an existing backlog. Instead, use in-market testers who are able to scale and available at the times and in the situations you require. Automated testing can run through some of these combinations at scale, but manual testers must be at the ready to handle exceptions — an impossible task to handle solely with internal testers.
One byproduct of the streaming media boom is that these companies are now tasked with becoming subscription management companies. This means added complexity to payment testing efforts, ensuring rate increases bill as expected, expired cards decline as expected, new forms of payment are supported, etc. Media companies, already hit with a high rate of churn, must do everything they can to maintain their subscription revenue at scale. Ongoing payment testing is one way to achieve that, putting a barrage of testers through the paces of real customer payment flows and quickly acting on the defects discovered.
Partner up to keep up
It’s unrealistic to conduct all aspects of payment testing in-house, as you need real payment instruments and transactions to validate. It’s also unrealistic to continuously manage this in-house for every market, every release, every payment flow and every customer journey.
This is where a digital quality partner like Applause offers tremendous value. Rather than waste resources on this unending task, Applause offers a million-strong community of digital experts matching your device, demographic, customer journey and payment instrument requirements. Applause takes an active role in sourcing and managing these testers to ensure your payment needs are met, on launch day and well into the future.
,
In 2022, Applause sourced payment testers in 162 unique countries, spanning thousands of test cases. These digital experts conducted primarily three types of testing:
Active testing, in which the tester executes a test according to your requirements in a physical or digital location.
Passive testing, where a tester’s payment instrument details are provided to an engineering team as an encrypted file.
Split testing involving the tester providing payment information then remaining involved with the test conducted with the engineer.
These different types of payment testing enable Applause to help validate payments throughout the SDLC — shifting left in engineering, prior to release, on release day and shifting right into ongoing production testing and maintenance. For media brands that care about strong revenue streams and frictionless subscriber payments, payment testing with Applause is a true differentiator — for you, or for the competition.
Talk to us today about how Applause can help meet payment testing needs.
10 min