Why Live-Event Testing Is an Imperative for Media Companies

Streaming media subscribers are up, even as streaming media budgets are down. Streaming platforms are seeking to stabilize their revenue in any way possible. That means being creative with revenue models, innovative with ad delivery and exhaustive with content options — within reason.

Here’s one emerging differentiator: live programming. At the beginning of the cord-cutting movement, subscribers often traded live programming for lower costs and convenience. But a growing demand for live programming is emerging, and streaming media providers are increasingly battling over these captive audiences.

In particular, sports programming presents an opportunity for streaming media providers, as evidenced by these recent finalized or rumored partnerships:

• Amazon inks a reported $1 billion annual deal with the NFL to stream its Thursday night games

• YouTube signs multi-year deal for NFL Sunday Ticket worth up to $2.5 billion

• Apple and Peacock acquire limited MLB broadcasting rights for roughly $115 million annually

• YouTube re-ups streaming rights partnership with MLB

• ESPN and Turner Media nab NHL broadcasting rights for about $600 million, including streaming games on ESPN+ and Hulu

• Apple secures exclusive streaming rights for MLS games for $250 million

• Netflix mulls bids on streaming rights to tennis and other sports

Even as the costs skyrocket for the rights to stream games, media providers are finding it a worthwhile investment — both to secure subscribers now and remain relevant into the future. If a media company can boost its subscriber pool by even a few percentage points because of a streaming rights deal, that’s several million subscribers added to the platform — and perhaps lured from competitors.

Investments in live programming, such as sporting events, come with a variety of benefits beyond an increased subscriber count. As they were in cable’s halcyon days, live-event ads will always be in-demand for brands wanting to connect with an engaged audience. Providers can deliver ads in a variety of innovative ways to a rapt base of subscribers. This programming also enables the option of promoting other content on the platform, such as an upcoming series launch or first-run movie.

However, unlike most streamers' content, live programming offers unique variables for testing. While some aspects of live programming can be validated before the show, there’s no way to test it all beforehand — and it requires a comprehensive effort to get the job done during the broadcast.

Let’s take a deeper look at streaming media companies’ push to compete and deliver on live programming.

The competitive landscape of live programming

Streaming media continues to capture an increasing segment of total TV viewership — and cable is on the decline. As Nielsen reported, even with a 222% increase in sports viewership on broadcast channels in the fall, cable continues to decline compared to the streaming genre. While sports accounted for 25.1% of all broadcast viewing in that time span, cable only captured 33.8% of total TV viewership, which shows audiences are migrating to other platforms to watch the teams they love.

With more viewers subscribing to streaming media, the pressure is on to deliver exceptional content, advertisements and experiences, both to capture and retain subscribers as well as to attract top brands as advertisers.

Live programming is becoming the norm on streaming platforms, and it goes well beyond sports — news and commentary, morning shows, talk shows, awards programs, musicals, and more. For the same reasons live programming was a treasured staple of cable programming, it carries value for a streaming media subscriber base looking to keep up with current events, both locally and nationally.

But it’s an area where streaming providers must proceed with caution. Not all live programming will find an audience, as Warner Bros. Discovery found out with the cancellation of its CNN+ platform just one month after launch. As with other digital properties, streaming brands must understand their customers and deliver great products.

As streaming brands expand into new markets each year, the drive to deliver live programming adds extra pressure on localization efforts. Subscribers want to see content aligned to their language, culture, norms and interests. For example, Disney’s Moana is known as Vaiana throughout the EU and in other parts of the world, due to copyright issues. Thus, a recognizable, world-renowned brand like Disney would want to be especially careful to refer to that heroine and her movie by the correct name in the correct language, both in live and original programming. Anything less risks damaging a brand that has a sterling reputation.

New-market launches can also present issues with payments, as different payment instruments are more popular in some markets than others, such as mobile wallets in APAC. Likewise, promotional offers might fail if those flows are not tested in-market — this was an issue we experienced with a customer of ours running a promotional launch of its service in Italy. This is important to keep in mind as live programming might involve audience participation, such as a vote in a talent competition, or limited-time offers.

Tie-ins between live programming and social media offer more opportunities for brand awareness and outreach. With influencers blurring the line between content creators and advertisers, media companies will find social media is another battleground in the nonstop battle for attention — and, thus, advertisers.

Considerations for testing during live events

Validating live event streams and the advertisements displayed during these events means testing on a massive scope. Not only must media providers cover a wide base of subscribers with many devices, OSes and carriers, but they must do so for customers located in various locales around the world.

For example, the NFL plays most of its games on Sundays, which means multiple games occur in multiple sites, each involving two primary viewership markets. However, it’s not enough to guarantee a sufficient stream for fans in those two teams’ hometowns. Expats might be watching the game in different countries, early in the morning or late at night, and streaming packages enable viewers from other markets to tune in as well, such as the case with platforms like MLB.TV and NHL Center Ice. That’s too much to test in-house, especially in-market during the live event.

To validate the content itself, testers must monitor the stream quality and verify that the playback controls work properly. If additional display elements are provided on-screen, such as in-game stats, they must also validate that those display and update properly. If excessive delays in the stream occur, especially if it points to a trend, like a particular device or location, that presents another issue media providers must examine. Issues like blank screens or ads not displaying are also common.

Ad delivery requires its own validation and presents still more challenges. Issues like repeating ads due to restricted demographics, competitive ads played in sequence, or only partial ads shown are common. Ads might also display for customers in the wrong market, or of the wrong demographic. Sometimes ads with expired promotions will display. Dynamic ads might serve the right market, but contain incomplete, inaccurate or null information, such as the nearest brick-and-mortar location being nowhere near the subscriber. These are all issues that can not only affect ad revenue but also create a negative subscriber experience — creating friction with streaming media providers’ two primary sources of income.

The timing of ad delivery during live sports is crucial, as viewers do not want to miss important moments in a game. Imagine the viewers’ angry reactions if an ad interrupted a critical moment in a playoff match. Another aspect of localization in live sports is language delivery, where some sports, such as the Spanish LaLiga, broadcast in multiple languages. ESPN transmits LaLiga games in both Spanish and English, which requires that ads be localized in multiple languages to reach different demographics within the same market.

VPNs can’t sufficiently replicate the experience of an in-market viewer, especially as streaming providers get micro-focused on ad delivery, enabling brands to advertise to more specific markets and demographics every year.

There’s another market segment to consider: the attendees of those live events. Increasingly, consumers are blending their in-arena and in-app experiences, creating a unique customer journey that presents still more challenges for digital products that go beyond streaming media. As our digital and physical lives intertwine, these present some interesting and exciting opportunities for live-event attendees, but will also require testers also validating from the event to ensure all functions and integrations work as expected.

Nothing beats the real thing

There’s no substitute for real testers with real devices in real markets. Device farms and VPNs might catch some issues, but that’ll be of little solace if you can’t capture a defect actively affecting a significant portion of live stream viewers. Streaming device fragmentation makes it increasingly challenging for testing teams to evaluate the quality of the stream, playback and ads, due to the many available resolutions, interactive camera views and closed captions availability, just to name a few. In-market testers trained to provide detailed reports and capture Charles logs help quickly identify and resolve these defects, especially when collaborating in real time.

Crowdtesting providers offer in-market testers to cover gaps in your internal testing coverage. However, nobody can match Applause for 24-7-365 testing coverage for live events. Our million-strong testing community enables us to find and train testers who reflect your target viewership, in the markets and with the devices you want. Many of our streaming media testers find pride in helping a media brand they recognize — or even love — deliver exceptional experiences for subscribers.

Test your live event streams and ads on a scale befitting the event itself. Invest in qualified in-market testing resources to quickly resolve bugs that would otherwise demoralize subscribers. Talk with us today about your streaming quality goals, and how we can help you achieve and exceed them.