18. What Is the Hardest Part of a Test Plan?

An approved Test Plan is essential to formal testing as it gives the scope and strategy for the whole testing effort…The IEEE Standard 829-1983 for Software Test Documentation provided a test plan outline that experience has shown to work for any testing effort and across all validation packages. (p.92)

 

The three types of computer validation packages are for End User Performance Qualification (PQ), IT platform system and infrastructure Installation Qualification (IQ), and software supplier Operational Qualification (OQ). All three types require an approved Test Plan and a full set of test documentation from Requirements and Trace Matrix through Test Scripts and Test Summary Report.

Developing the scope and strategy for testing is often the hardest challenge for new validation teams. This is where the Requirements document becomes a life saver. For the software supplier the Functional Requirements and Software Design Specifications (FRS/SDS) define the OQ testing scope. For IT teams a system platform and infrastructure requirements specification (PRS) serves the same purpose. For end users the work process based user requirements specification (URS) defines the scope of testing in the PQ Test Plan.

Once the requirements document has defined the scope of testing the next big hurdle is developing the approach to be taken. Software suppliers have many automated tools to help them do this, but end users are left more to their own devices to figure this out and often have little experience to call upon. Experience has shown that there are three phases to such testing.

• Phase 1 is checking Work Area Preparedness – SOPs, Work Instructions, User training records, System manuals, Online Help, System support
• Phase 2 is System Administration – Security and audit trail checks, user/role privilege setup for testers, database build for testing
• Phase 3 is Work Process Stream Execution – Follows the whole system work process stream using three characteristic examples of work in order to exercise all the requirements.
  • Vanilla Example – all normal data, normal actions, normal process
  • Chocolate Example – problem data, incorrect actions, process errors
  • Strawberry Example – disaster recovery, security challenges, other work process issues

In the view of the IEEE Standard 829-1983, each phase above would have one or more Test Cases that organize multiple test scripts into related work flow activities.

The phased testing approach is best shown as a table in the Test Plan. Working through this table and the Trace Matrix to include all testing up front is extremely important. It provides a big help to efficiently organize the writing and execution of test scripts and ensure that all the defined requirements get tested. This is a tedious process and requires large chunks of time with your best people who know the work process in detail. The reward is a testing strategy that passes audits and ensures the system is fit to its defined purpose. Never sign off on a Test Plan without a completed Trace Matrix that shows just what testing will be done for all approved requirements.

 

Next Month: Why have a Test Summary Report?

No formal testing effort is complete until a comprehensive Test Summary Report has been written and approved. Often validation teams think of a Test Summary Report as one last documentation insult to their precious time and energy. (p.94)