Previously in this series, we talked about how we can take a whole-team approach to agile testing. One of the most common challenges in testing I’ve seen over the years is around communication and establishing a team-wide understanding of how code flowed through the system. Quite often, teams lack a consistent understanding of CI/CD workflows, including what tests were being run and when or even what the tests were for. This lack of common understanding kept teams from reaching their full potential. In order to consider testing as a team in continuous delivery, we need to address this communication gap and set our strategy together.
How Does Code Flow?
You may be asking, why do I need to know how code flows? I’m just a <insert your role here>. Many teams I’ve worked with that considered themselves “Agile” still remained siloed. Developers did developer things and submitted their pull requests. Testers only tested (or wrote automation scripts). Some team members may have known when a test was triggered, but not what happened after that. Perhaps they didn’t even know how the code flowed through the system at all. But, how else can you know where to test, if you aren’t clear on what happens once you create a pull request to get value into your clients’ hands?
One exercise I highly recommend is drawing your pipeline. From the time you make a commit to deploying to production, what is happening? When does your code merge and where? Do you promote straight to production, or to a staging environment? Take some time to draw it out. First, I recommend drawing out current flow, and then your ideal pipeline. This can include manual, automated, or both. (As Lisa Crispin always calls out—even if nothing is automated, it is still a pipeline.) NOTE: this may look different for each organization.
Image: Simplified code pipeline (not including testing)
But I know we aren’t just committing our code without testing… Let’s talk about how we think about our testing in the CI/CD workflow.
Planning Feedback Loops in Our Pipeline
Think about the pipeline you just drew. It’s time to think about the steps our changes need to go through before they can be deployed to production. This can look different for every company and team. As we progress our code through the pipeline, how might we increase our confidence at each stage along the way? Why… testing! Testing is a central part of the CI/CD workflow.
Now, in your pipeline you just drew… what kind of feedback (testing) might you need in order to determine if your code should continue to flow into production? When do you think you need that feedback? Add this testing to your pipeline.
Image: Simplified code pipeline with testing steps
The goal here is to have a common understanding and vision of how our code is released, and the testing we want in place to gain confidence. You may not have all of the pieces in place—that’s OK. I was introduced to the concept of a strawman last year. We wanted a high-level vision of our CI/CD workflow. In our case, we were looking at how our code got to release and the testing we wanted in place. We knew it wouldn’t be an end state, and, as we tackled each piece, we knew it might change. We knew this strawman would encourage and guide iterative improvements as we developed each piece. As a team, this is a great way to work together towards that vision, and communicate constantly about what’s working, what’s not, and how this pipeline might change as we develop it.
There are so many ways your team can go about doing this! My favorite is a table activity, but that’s not very social distancy :-( But here are some ideas to get you started:
Any collaborative virtual whiteboard (like Miro). Make some stickies like the printable cards and get going as a team.
When the teams I’ve worked with have taken the time to draw this out together, it was really useful. This marked the first time I really heard in-depth questions emerge and teams talking together to decide what needed to happen before we felt confident going to production. There were moments where we questioned if we needed certain things, or where we could think about running things simultaneously. Sometimes we would draw current state, then draw ideal state together.
We’ve all seen different pipelines, with varying levels of feedback. What worked for one code base may not work for another. It’s important to leave judgement at the door and be open minded. The key is to communicate as a team, set your strategy, and dive in. (Note: It can be really hard to get everyone on the team working together on this step. Even if you simply pair up with someone else, this can help generate questions to go back and ask the team about, which is still beneficial.)
Next up? We’ll talk about how we establish feedback loops that enhance delivery pipelines, including an introduction to a test suite canvas that can be really useful in these efforts.