Leading Agile Testers and Testing in at Scale Contexts

Mary Thorn, Director of Quality, Ipreo

Over the past 20 years, I have had the opportunity to work at many startups and created several QA/Test departments from scratch. Over this same time period, software development methodologies have changed. And I’ve learned that creating a QA department from scratch in a waterfall world was a lot easier than in an agile world.

In a waterfall world, I had nearly complete autonomy to the processes we put in place, the tools we used, and the people I hired. Things changed radically for me when I transitioned to agile. In agile, my testers are now part of Scrum or Kanban teams. Among other things, the testing practices I want to implement have to be taught to the entire Scrum team. The tools I want to use need to gain buy-in from the whole team. And the people I want to hire have to be vetted and approved by the team. Needless to say, it has been a learning experience in how to lead agile testers and agile testing in at scale contexts.

My first major learning is that consistency at scale matters between testing processes on Scrum teams. When I say “at scale”, it implies leading testers on 10 or more Scrum teams. What I have learned is that while we want to keep Scrum teams together as long as possible it is not always an option. Having the ability to move testers from one Scrum team to another, or swap out testers because they just don’t “fit” in their team, is a necessity.

  ​You have to have Test Managers in your programs to ensure your mission and vision is clearly communicated, well-understood, and executed  

To achieve this consistency, you need to set standards across Scrum teams regarding testing. This could mean at minimum the teams have a risk based test plan template and standards, they write test cases in the same tool as all the other testers, and they do automation is the same framework. Most of these items should be included in your organizational definition-of-done, but how the DoD is implemented is what I mean here. It’s important in at scale contexts for it to be consistent across all the Scrum teams.

My second learning is that you have to have Test Managers in your programs to ensure your mission and vision is clearly communicated, well-understood, and executed against. In one of my past organizations, there were 120+ testers and 40 scrum teams when I came in with just two Test Managers. Most of the testers reported into Product or Development Managers. From software testing craft and discipline perspective, I challenged the organization that was not the right model. I recommended they invest in Test Managers who would train and upskill their testers. Managers who would set the mission, vision, and roadmap for all testing in the organization and who would also partner with the PM and Development Managers to remove any testing impediments for the individual Scrum teams. And Test Managers who would lastly raise risks and engage in honest and open communication with senior leadership when needed.

We also realized we needed to hire a few more senior testers who could balance out the Test Manager when the Test Manager could not be in most of the calls. The Senior Tester would take on most of the coordination of activities like regression test phases (Hardening), as well as, help with raising Testing Technical Debt stories.

Lastly and most important learning is that as the Director of Testing at a large agile organization my entire view of my role changed. I went from knowing what was happening across all my Scrum teams to the point of only hearing it from theTest Managers. I had a hard time letting go. I also had a hard time figuring out how to get my vision of what good agile testing looks like rolled out across all 40 teams. Then one day I finally realized, why don’t the Test Managers and I operate like a Scrum team too? I could serve as the Product Owner; the Test Managers would be the team members. My stakeholders would the Product Managers, my boss the CTO/CEO/President, and Development Directors.

My next step was to create a backlog of all the items I wanted consistent across all the teams. The Test Managers and I started a sprint where we wrote Epics and broke them down into stories related to our consistency-based testing strategy. Over the first year as a group we moved together. If we decided that everyone was going to start writing their test cases in a new tool, we all focused over the next sprints did this together. The benefit was that we could learn from each other, we could hear the issues that we all were running into and share how we fixed them.

My Test Managers were awesome in that they would even call me out even though I was their boss when my stories were not up to par or I was not pulling weight. I also learned that every quarter I needed to update my roadmap and socialize my progress against the previous roadmap and what was next. I needed to sell the value and promote the change and in metrics show the progress on how quality was improving. My roadmap would then feed down to my Test Managers and then down to the individual Scrum teams.

In summary, agile testing in at scale contexts is hard. What I have found is that if you have a plan, show progress towards it, iterate on it, and as a test leadership group triangulate towards the same direction you can significantly reduce the challenges. Also, as a leader you have to continue to work with the product and technical leadership to ensure they understand the practices you are trying to improve and the “Why” behind it and get them to support you and your shared journey towards ever-improving testing.