How to Achieve the SAFe Core Value of Built-In Quality

Many large-scale, Agile organizations struggle to implement the SAFe core value of built-in quality. While lacking resources and expertise are contributing factors, the main problem is that teams working in scaled Agile often underestimate the level of commitment required to meaningfully build in quality in a scaled Agile context.

Matt Vance, Quality Manager for myAudi, shared this message at a recent Applause webinar. According to Vance, failure to successfully implement built-in quality is just as much a mindset issue than a question of logistics.

“Teams are constantly pushed to deliver business value,” explained Vance. “Unfortunately for our train, and I can imagine for a lot of trains, testing is the first thing to be sacrificed and really living SAFe’s core values is one of the main challenges we have been dealing with. Built-in quality, transparency, and alignment aren’t just empty words; they need to be drilled into teams by senior management. If you start with the wrong mindset, it can have a long-lasting effect.”

To get in the right mindset, organizations must understand exactly why the SAFe core value of built-in quality is crucial for Agile development at scale.

Why the SAFe core value of built-in quality is so important

To keep up with today’s rate of technological advancement, companies need to maximize release velocity. How fast they can deliver new features, however, depends on the quality of the solution. This is because teams that spend more time during the development process to ensure a high standard of quality at each stage ultimately spend less time reworking and fixing products later on. As is stated in the Agile manifesto: “continuous attention to technical excellence and good design enhances agility.”

SAFe recognizes that quality cannot be forced onto a product once it is already finished; it must be built in. As a quote on the SAFe website reads: “Inspection does not improve the quality, nor guarantee quality. Inspection is too late. The quality, good or bad, is already in the product.” In other words, even if teams were prepared to spend lots of time recalling, reworking and fixing defects, the product would still not reach the same level of quality as a product that was designed for quality from the get-go.

Especially in Agile software development, in which products undergo multiple iterations, quality must be built into every stage of the development process to ensure agile teams do not introduce new errors with each change. The larger the organization, the greater the potential cumulative effect of even the tiniest bugs — this is something the myAudi team does not take lightly. “Our architecture is pretty complicated with the different teams and trains,” said Vance. “We have backends from different companies, the vehicle, the app and the website. Even when one small change breaks one of the systems in the backend, that can cause a complete failure in the eyes of the customer.”

Large-scale organizations that treat quality as an afterthought risk finding too many issues too late, when bugs may have become so integrated that they are difficult to rework or fix without overhauling entire features at great expense.

How to achieve the SAFe core value of built-in quality

Without understanding exactly why the SAFe core value of built-in quality is so important for scaled Agile development, teams are unlikely to make it a priority. However, really wanting to build in quality and getting into the right mindset is just the first step toward successful implementation; it doesn’t solve the problem of how.

Like many high-level frameworks, SAFe provides theories to follow without detailing how to implement them in practice. Marcus Schmidt, solution QA consultant for myAudi at Applause, believes that this is a key issue that deters organizations from committing to SAFe. “While SAFe is great for providing guidelines, for most projects, the devil is in the detail,” explained Schmidt. “The framework does not provide a clear answer on how quality assurance needs to be implemented on a detailed level. It doesn’t deal with operational hurdles we face each day, so conflicts of objectives like time pressure, lack of expertise or resources are excluded from its theoretical considerations,” he added.

To work out how to implement SAFe in a way that makes sense for your organization, Schmidt recommends first working out where teams currently stand in terms of quality and where they could go. Many groups start with Test Maturity Model (TMM) interviews, in which each team member answers questions on their current testing methods and challenges in areas like exploratory, in-sprint and regression testing. Using this information, solution management should agree upon success metrics and set up BI dashboards that then serve as a benchmark for quarterly PI planning meetings.

What the SAFe core value of built-in quality looks like at Audi

myAudi worked together with Schmidt and the rest of the Applause team to create a clear SAFe implementation strategy and grow its built-in quality practice, using the information gathered in TAMM (Test Automation Maturity Model) interviews as a basis.

Rather than seeing SAFe’s unspecificity as a limitation, the myAudi team saw it as a license for creativity, an opportunity to adapt the framework to their team and resources. One of the first things it did together with Applause was to create a central quality assurance team to provide the strategy, tools and directions to each train within the solution, as well as take responsibility for end-to-end testing. This is not something SAFe specifies, but proved crucial for myAudi to steer its quality efforts across Agile teams and trains and ensure a holistic approach to overall solution quality. With 10 full-time employees, the central testing team plays a considerable role in making the SAFe core value of built-in quality a success.

Once the central team was in place to oversee built-in quality at the solution level, myAudi and Applause also worked to improve the approach to built-in quality within individual myAudi Agile teams. One key improvement was the addition of test analysts to each team. These Applause QA experts are responsible for planning and executing tests, as well as sharing learnings from PI planning meetings with tests analysts in other teams. Test analysts also collect information about upcoming feature changes and assess required amendments to testing, which they report back to the central QA team.

The myAudi solution team also knew that true built-in quality could not be achieved without finding a way to integrate the end user perspective in testing. Even with the central testing team overseeing testing strategy and test analysts carrying out tests on the ground, without the end user perspective, the myAudi solution still experienced organization blindness. Working day in, day out on the same solution leads to an over-familiarity that can cause key customer use cases to be overlooked — and the only way to avoid this is to get real testers with real devices to provide unbiased feedback.

Where automation fits into the SAFe core value of built-in quality

Greater automation does not equal higher quality. Automating tests is only effective for repetitive, checklist tests that do not benefit from real user input. The best SAFe Agile teams take a hybrid approach to automation and manual functional testing, in which humans step in to test potential problem scenarios not previously considered.

Making built-in quality a success is a long-term commitment

Part of making the SAFe core value of built-in quality a success is understanding that it cannot be implemented overnight. No one organization or product is alike and designing the right approach takes time. Even if organizations spend a whole year setting up built-in quality within SAFe, they will still tweak their approach in the years to come.

For practical advice on how to implement SAFe in your organization and more information about how Applause works with myAudi on this topic, read our eBook: How SAFe Organizations Can Achieve Built-In Quality.