“Leaving anything until the last minute is never a good idea,” advised James Mortlock, Lead User Acceptance Testing (UAT) Manager at Vodafone, in a recent Applause webinar. Companies that work in Waterfall lump all testing onto the end of the software development process, which results in major issues being discovered too late — pushing back timelines and driving up costs.
That’s why Vodafone’s UAT department, under James’ leadership, decided to adopt an Agile setup and shift testing left. Here’s what James had to say about Vodafone’s transition, complete with advice on how to implement Agile UAT in your own organization.
Waterfall vs. Agile
“When you leave testing until right at the end of the SDLC, you have no idea what kind of bugs you will discover, nor how complex they will be. When faced with these bugs, you then have to decide whether to launch anyway, and risk reputational damage, or delay the delivery of the software. Given how much pressure there usually is to get a product out, delaying may not even be an option. This is the consequence of Waterfall testing,” James said.
“Agile allows you to identify issues early on, so you can fix them more easily, quickly and cheaply. Instead of taking on monolithic, 6-month projects, you start thinking in terms of hours or days, testing and releasing features in small chunks so that you can provide customers with value much more frequently. Quality also increases because you focus on one feature at a time and are much better equipped to respond to changing customer requirements.”
Want to move to Agile UAT? Don’t wait for sign off
James explained different approaches to making the shift: “One way to transition to Agile is to build a case, get all your data and information together, find a sponsor, pitch it, get approval, and then create a roll out plan. Usually, you’d start with one team, see how it goes, start rolling it out and then incrementally increase the number of teams. Doing it this way, if it is beneficial, takes a long time to generate results.”
“Or, you can do it how I did it at Vodafone and just rip the plaster off. Just do it. If you have hired someone to do a job, you should trust that they know what they are doing and let them get on with it. That was my approach to this,” James said.
“That said, saying you are going to rip the plaster off is one thing; making sure you can deliver benefits is another. You can fail fast, but deliver value and show people that you don’t need to do something one way just because you have done it like that for years. For me, it was about forgetting the old-school rulebook and working out what we really wanted to do, which is delivering quality and making sure that users have got what they want, but doing it faster.”
Getting started
At Vodafone, James said, “we used what already existed. For example, instead of writing test cases and having endless test artifacts, we embraced the Agile idea of having working software above comprehensive documentation and used our user stories as a bible for everything. We have our acceptance criteria in there, which effectively act as our test cases. We are not delivering work items or code, we are trying to build a story for a user to go through. So, we use the user stories. This works especially well for UAT because it is not proscriptive, but allows developers to do what they are good at, namely finding ways to solve problems in their code.”
James continued, “we treat the discussion piece of Azure DevOps as a requirements and traceability matrix. This is something I am very keen on. In traditional UAT, there is a lot of emphasis put on tracking whether X belongs to Y requirement or test case match up. In the discussion piece, we have our structure, features, user stories, etc. All of our notes are there; developers and product owners reply on the thread and it keeps all information we need for each piece of functionality. The governance is there, the traceability is there — we’re removing as much documentation as possible. This frees up time to build in things like crowdtesting.”
Crowdtesting adds the “user” to user acceptance testing
“Crowdtesting helps you see the wood in the trees,” according to James. “Once you embrace Agile as a new way of working, crowdtesting is another new avenue you can explore. Global teams can be built on request, with testers that are either familiar or purposefully unfamiliar with your products. Instead of having monolithic QA teams that aren’t really doing much for parts of the year, you can scale crowdtesting teams as and when you need them while building predictability into your workflows.”
James said, “if I was to give anybody any advice on making Agile UAT a success, it would be to think more like a user and less like an engineer. Just like you, users put their trousers on one leg at a time, so it shouldn’t be that difficult to think like them!”
Capturing feedback early in the development process is crucial, according to James: “You don’t want to launch a product and then get the feedback, you want to simulate that feedback in sprint. And users don’t write test cases. If you compare any user testing that you do to your bank of test cases, they rarely match and you usually have to use multiple test cases to hit one line from user research. A lot of the bugs that my team raises go against the usual UAT viewpoint that bugs need to match a requirement.”
Watch the full webinar with Vodafone below.
5 min