RPA Tools for Testing: Next Big Thing or Bad Thing?

Robotic process automation (RPA) is an established technology in workflow automation.  Now we’re hearing more about the application of RPA for testing. Is this a good idea or a bad idea? How does RPA fit into your continuous testing goals? 

First, let’s define what we mean by RPA. From Wikipedia:

In traditional workflow automation tools, a software developer produces a list of actions to automate a task and interface to the back-end system using internal application programming interfaces (APIs) or dedicated scripting language. In contrast, RPA systems develop the action list by watching the user perform that task in the application’s graphical user interface (GUI), and then perform the automation by repeating those tasks directly in the GUI. This can lower the barrier to use of automation in products that might not otherwise feature APIs for this purpose.

RPA Reinforces the Evil Ice Cream Cone

If you’re wearing your continuous testing, shift-left hat, then the Wikipedia definition probably gave you pause. When applied to testing, RPA emulates and automates how a manual tester interacts with an application. In that way, RPA helps you build, expand, and reinforce the evil ice cream cone—an anti-pattern where more testing is done at the GUI level and less is done earlier in the lifecycle.  

The more you rely on RPA tools, the more testing will be done as a time-boxed event at the end of the SDLC. This means: 

• Problems are not found until the end of the release cycle, when root cause is harder to identify and defects are more expensive and time consuming to fix. Waiting until the end also increases the possibility that defects will escape to production. 
• Testing isn’t shifting left—so quality will continue to be a bottleneck. Sure, you can do all your manual processes faster, but it’s still an extra phase, rather than a continuous process. If you find a problem, you still have to start all over again (find, fix, retest) and with a larger “blast radius” of where the root cause of the problem could be. 
• Agile, DevOps, and continuous delivery initiatives will all continue to struggle. You can’t achieve volume, velocity, and quality unless you break down silos and move past legacy methodologies. 

Caption: Evil Ice Cream Cone of Testing

https://www.thoughtworks.com/insights/blog/architecting-continuous-delivery

Automation can make ice cream pretty fast, but it’s still ice cream.

However, when you shift quality left (the pyramid) errors are found closer to the source, when root cause is easier to identify and addressing the problem requires a minimum of rework. Using data from the pipeline and production (shift right) provides insight into where inefficiencies and areas for improvement lie. Visibility into the pipeline and data-driven insights encourage collaboration and bring teams together. This is continuous testing and this is how you can address some of the biggest barriers to DevOps, Agile, and continuous delivery maturity. 

In the words of someone who knows quite a bit about continuous delivery… 

“Tests that run end-to-end through the UI are: brittle, expensive to write, and time consuming to run.” 

— Martin Fowler

Not even vendors who offer RPA tools *for testing* are sold on the technology *for testing*…. From the GM of RPA at Tricentis: “Even this simple example [see the blog post] exposes some of the many software testing complexities that RPA tools just aren’t designed to address. RPA tools are built to automate specific tasks within a sequence. Software test automation tools are designed to measure the resilience of a broader sequence of tasks.” 

Yes, that’s what he said. RPA tools aren’t really designed for test automation. It’s not the same thing. 

Does RPA Have a Place in Testing? 

RPA vendors would like organizations to embrace this new use case—new uses equals more business. And vendors are familiar with the struggles organizations have had with RPA. But does RPA have a role in your testing toolbox? 

https://www.horsesforsources.com/RPA-teenage-romance_011218

Umm, maybe? RPA tools can be effective for allowing business users to test in certain cases. Jesper Ottosen presents some examples here that include UAT for implementations of standard systems like SAP and Dynamics 365, where the organization does not “have the need or know-how to test configurations or code below the GUI.”  So in that edge case, yes, maybe you’d want to use RPA to speed up the process when you’re doing an implementation, and you only have business users available to test, and you only have the need to test at the GUI level. 

Also know that focusing on adding new technology to legacy processes only detracts from achieving shift-left goals. RPA is brittle, it’s going to take time and resources (people, money) to maintain, and it’s putting a band-aid on a problem that can only be fixed by true shift-left continuous testing. 

The Pyramid is a Better Choice

In almost every case, you’ll want to follow the pyramid model to achieve quality with velocity and scale. Shift testing left and embed testing into your CI/CD toolchain so the testing is orchestrated in the pipeline. Then, monitor at the API, functional, and application level to ensure ongoing reliability and performance.

Continuous testing is the only way to address quality blockers and achieve DevOps and continuous delivery success.