New AQS™ Scores Provide Clearer Picture of Software Quality

The Applause Quality Score™ (AQS™) rating gives you a measure of quality from one release to the next — like a credit score for software builds. Now, AQS is going beyond bug data to provide a more nuanced look at the quality of your builds.

Available now, AQS will incorporate test case data to offer a deeper, holistic view of digital quality. This customized insight helps you pinpoint where and how to improve testing strategy with the help of an Applause expert. With these different scores, Applause customers can quantitatively evaluate their digital quality at a build, release and product level, and gain confidence that they are effectively testing their digital products and can make data-driven release decisions.

“By expanding the Applause Quality Score algorithm and enhancing the activity dashboards, teams now have an even more reliable view of their build quality, said Kristin Simonini, VP of Product at Applause. “With this latest evolution of AQS, we can provide even more meaningful guidance to our customers.”

The Product Overview tab displays past and current AQS ratings, with a trend line to gauge whether quality has improved or slipped. You can also view all three scores, indicating quality based on exploratory testing, structured testing and overall.

AQS will provide three different scores, depending on which Applause features you use.

• Structured testing score (Test Cases). This score reflects the quality of your build based on the results of structured test case runs. The score examines details for test runs, including which passed or failed, for which components and the severity of failures, to provide a metric of functional app quality. With test cases covering critical user flows, a poor score here can indicate when you’re suffering from regressions and customers will have poor experiences that impact the bottom line. The Test Cases score is only available to customers who use Applause for structured test cases.

• Exploratory testing score (Issues). The exploratory testing score assesses issues testers discovered and how they are triaged to make informed release decisions and potentially improve testing strategy. This score incorporates data such as bug priority relative to the duration of testing. As exploratory testing incorporates testing unexpected user flows that a structured test case can’t anticipate, this score gives you perspective on the ways customers will use your app. To receive an Issues score, you must conduct exploratory tests with Applause.

• Combined (Quality Score). For users with access to both the Test Cases score and Issues score, the Quality Score calculates a weighted score relative to the testing profile of the product or the amount of structured or exploratory testing under assessment. Thus, the Quality Score provides a blended assessment of quality.

Each of these scores runs from 0 to 100 — the lower the number, the lower the quality of that particular build. The higher the number, the better the assessment of your application quality. You can toggle between the different scores and displays.

You can assess these scores along with a Confidence Level displayed in the dashboard, which looks at the scope, scale and history of testing in the platform to provide a measure of assurance in the calculations behind the AQS rating. Confidence Level weighs details about the maturity of the crowdtesting implementation and the reliability of the structured/exploratory testing data in its calculations. The higher the confidence level, the more data supports the AQS scores.

As part of this addition, you will also be able to view real-time bug data and/or test case data within the AQS dashboard. They simply toggle the Issues or Test Cases button, then drill down to the data they want to find.

The Quality Score by Build tab gives details about your Issues (bug severity, bug value) and Test Cases (number of tests, scope, executed, etc.) Sub-Scores.

AQS also displays quality trends over time, enabling you to see how quality has improved or regressed. Trend data can help organizations pursuing continuous improvement, as it provides consistent benchmarks to meet or exceed. This AQS trend data provides valuable insight for organizations in two key areas:

• new builds of a single release, where the organization can make sure AQS ratings increase as the release candidate improves;

• trends between different released builds, which can fluctuate in rating due to variations in release scope and complexity, but must still be tracked and ideally reach a quality benchmark.

“With visibility into their quality trends, customers can take immediate action to address problems and remove the risk associated with low-quality releases," Simonini said.

Learn more about AQS and the Applause Product Excellence Platform today.