Establish Continuous Mainframe Testing of Applications

    Today, mainframe application development and mainframe testing architectures are getting a new lease on life. Old designs are being revamped to keep up with modern-day business needs and an ever-changing marketplace. A key driver is the digital transformation underway in today’s application economy—characterized by frequent code changes to address rapidly changing requirements and the demand for quality user experiences.

    Now it’s time for mainframe testing programs to keep up with these new marketplace realities. One important example of why: The proliferation of mobile/web applications has driven exponential growth in the use of mainframes as a system of record. And that makes continuous, preproduction testing on the mainframe an imperative. It is simply untenable to try to manage for spikes in processing consumption on the backend. Traditional mainframe testing approaches need to evolve so that you can test earlier and often in preproduction—ensuring the quality performance of your apps and any incremental code changes your team makes.

    Here are three important best practices that can help you launch a continuous, pre-production testing program to support your mainframe environment:

    1. Embrace Open-Source Technologies for Mainframe Testing

    While most mainframe application development is waterfall-based, a paradigm shift is underway. Instead of monolithic greenfield projects, the emphasis is on brownfield development of enhancements and fixes. Faster and more modern Agile delivery techniques are becoming the new norm.

    Open-source tools are aiding in the transformation by demystifying mainframe application development and testing that used to cater for legacy COBOL programming and green-screen transactions. Developer teams now can code in their local IDE of choice, with no proprietary languages to learn. And the same is true of testing. New open source-based test automation frameworks allow you to easily write and execute small-scale tests from a local machine, and then seamlessly reuse the same tests at scale via the cloud to accelerate test automation as part of a continuous delivery pipeline.

    Open source-based testing can be a welcomed relief for resource-limited test centers of excellence (CoEs). You can extend test coverage to mainframe development teams and respond to growing dev-test demands. You can adopt a single, integrated test platform that concurrently supports green-screen testing of mainframe application transactions on-demand across remote terminal emulators. In addition, you can do automated performance, functional, and API testing at scale across multiple mainframe projects, each with its own dedicated workspace. You get a centralized view across projects, as well as drill-down reports and analyses for each workspace. What’s more, these new open-source testing tools have zero platform dependencies, so they will work seamlessly with your legacy test platforms and processes.

    2. Use Virtualization to Overcome Mainframe Testing Constraints

    Once you’ve adopted open-source testing tools, how will you use them to determine the impact of upgrades and fixes on processing capacity and the performance of distributed applications? How will you determine whether code changes will trigger a cascading failure and prevent return to a steady state? To do so, you need to test in representative preproduction environments. But critical systems may be unavailable, where distributed composite applications may not match up well with your mainframe systems.

    The answer is service virtualization, which lets you model the core business logic in your mainframe platforms and how they integrate with in-house and third-party systems. Your production environment continues as normal in one virtual space, while application testing is done in another. By identifying the boundaries between the components involved (the application, database, message queuing system, and so on), you simulate production conditions, using agent-based virtualization to record requests and responses. There is no modification to your CICS, DTP command, or target programs. Also, service virtualization supports “negative scenario” test simulations so you can explore error conditions, data corruption, elevated latency between components, a slow-to-respond backend, and other “what-ifs” that are difficult to reproduce and test in real life.

    Ultimately, virtualization facilitates your shift-left approach—testing and validating mainframe application development sooner, at a point where issues are easier, less disruptive, and less expensive to resolve. You get faster build-and-test cycles, higher-quality applications, and lower costs.

    3. Leverage Test Data with Sensitive Information in Mainframe Testing

    Mainframes are typically the system of record for core business logic and data—making built-in security capabilities a must. But mainframe application testing to virtualized services requires access to that sensitive real-world information and production data.

    To date, there have been two approaches for addressing this dilemma. One way is to route copies of production data into a test environment as data is generated, while making sure personally identifiable information and other sensitive data is deleted or anonymized. The other is to capture test data from the production environment and replay it asynchronously in preproduction. Both approaches have limitations that result from the slow, complex, and manual way that test data is created and provisioned.

    In cases where system components already exist, virtual data for preproduction testing is created through manual record-playback with cross-system availability dependencies, which results in delays. Exposing production data in preproduction environments increases the risk of a data breach and compliance penalties. In the event relevant system components do not exist, request/response pairs of sample data are created manually. These scripts are often complex and time-consuming, and they fail to reflect realistic functional behavior or performance, current message definitions, API specification changes, outliers, or future scenarios that require more rigorous testing.

    Clearly, a more sophisticated approach is needed to generate virtual data on demand. One that’s free from cross-system dependencies and constraints, while reducing infrastructure costs and project delays. You need to generate virtual data for service components, to cover a full range of possible scenarios without manual creation or costly maintenance. Alternatively, choose to create virtualized data synthetically from scratch, which complements new and existing data. Using virtualized data to complement production data, teams can access the latest pre-production environment, to fully test code and detect defects in parallel.

    Continue using production data securely without sensitive information leaving your private network. Avoid compliance concerns and risks while staying on budget, by first installing a physical or virtual test server on premises. Make use of a Docker-based private agent to generate test loads behind your firewall and within your private network or cloud. The agent “listens” and triggers application performance tests across the firewall using the load traffic you generate. Retrieving predetermined anonymized test configurations from a SaaS test platform in the cloud, this same proxy server matches them with the relevant data on-premises—indexed for use as tokens. Test scripts are then executed using the tokens and loads you’ve generated behind your firewall. All aggregations, manipulation, and advanced logic are done in the cloud. Anonymized reports and analysis are sent back to your proxy server for decoding—allowing you to view the test results.

    Select the Right Tools for Mainframe Testing of Applications

    This new way of testing keeps sensitive data, test scripts, and logs behind your firewall—including all URLs, APIs, command names, and arguments. There are many advantages. You can keep information secure, while flexibly scaling your test program on demand. You can equip teams to test code fully in your latest preproduction environment of the mainframe and detect defects earlier using virtualized data—in shifting your mainframe application testing earlier in the lifecycle with open source technologies. Ultimately, the right testing tools can help you jump start your continuous, preproduction application testing program.