19. Why Write a Test Summary Report?

No formal testing effort is complete until a comprehensive Test Summary Report has been written and approved. Often validation teams think of a Test Summary Report as one last documentation insult to their precious time and energy. (p.94)

Weary testing teams sometimes argue that they have a formal requirements document, a Test Plan, a Trace Matrix and a huge number of Test Scripts and Result Logs, surely that should be enough proof that they worked hard to test the system. Working hard is not the issue. Working smart and in compliance is the issue. Validation teams need to think of their Test Summary Report (TSR) as the billboard by which they sell management, auditors, and inspectors on the excellent job they did in formal testing of a regulated system to assure its reliability in performing its intended function and compliance to GXP.

The Test Summary Report provides the opportunity to scope the testing effort:

• Who were the testers, witnesses, and script writers – what were their names?
• What were the dates of the testing?
• Where were the testing sites, e.g., all at one site, multiple sites from other countries, from employee remote VPN logins, from client or partner sites…?
• Were there deviations from the Test Plan with Test Scripts or Test Sites added or deleted and why?
• What were the issues arising and how were they resolved during testing?
• When was each task listed in the Test Plan completed?
• How many requirements were tested and does the Trace Matrix show that each requirement was tested by at least one Test Script?
• Which Test Scripts challenged 21 CFR Part 11 and EU GMP Annex 11 requirements for the system with negative testing actions?
• Were all system modules/functions tested? If not, why not?
• Provide a table to identify each failed Test Script with a description of why each failed, how it was repaired, and the retest results after the Fix cycle.
• What tools were used for issue tracking, Test Script creation, execution, and Results management?
• What is the recommendation to Management for release of the system for GXP production use based on the testing?

A comprehensive Test Summary Report along with examples of some failed Test Scripts, the Trace Matrix, and Approved Test Plan and Requirements document should be enough to satisfy an auditor or inspector without spending hours scouring through test results. Without a comprehensive Test Summary Report, an auditor or inspector could spend hours going through test results and will invariably find something amiss from results written in pencil to skipped or missing printouts, etc.

Next Month: What is a Platform Requirements Specification (PRS)?

Separate from the URS, a PRS should be defined to describe the users operational requirements, e.g., 24/7 access by telephony, the software supplier’s technical requirements, e.g., database and infrastructure specifications, and IT operational needs, e.g., fit with other applications, as well as ongoing support of the software application on other systems in the organization. (p.153)