Testing Computers Systems for FDA/MHRA Compliance

David Stokes

• Publisher: CRC Press
• Year: 2003
• eBook ISBN: 9781135505974
• Topic: Medicina
• Subtopic: Farmacologia

There is no substitute for extensive testing when it comes to IT systems. Recognition that problems are easier and cheaper to fix before the system is in use (rather than after), has turned testing into a cost-effective tool. However, when developing computer systems for pharmaceuticals manufacturing, testing to meet regulatory requirements adds an

CHAPTER 1
Purpose

The purpose of this guideline is to:

• Demonstrate the value of a systematic approach to computer systems testing (‘why we test’).
• Provide a pragmatic method of determining the degree of testing necessary for any given system (‘what to test’).
• Provide a detailed guide to the recommended contents of computer systems test specifications and how to produce these in the most cost effective manner possible.
• Show where computer system testing sits in the full validation life cycle and where the tests sit in relation to the overall project.
• Provide practical advice on how to conduct computer system tests (‘how to test’).
• Provide guidance on the use of automated test tools in a compliant environment.

CHAPTER 2
Scope

2.1 What This Guideline Covers

This guideline covers the following areas:

1. The cost/benefits of conducting an appropriate degree of system testing.
2. A practical approach to determining exactly what is ‘an appropriate degree of system testing’ and how this can be justified (and documented) from a regulatory perspective.
3. The life cycle management relating to the development of Test Specifications and the conducting of these system tests.
4. The roles and responsibilities of those involved with the development of Test Specifications and the execution of these system tests.
5. The relationship between System Test Specification(s) and other project documentation.
6. The relationship between the system tests and other aspects of the project implementation.
7. Recommended content for inclusion in System Test Specification(s).
8. A traceability matrix defining how the System Test Specification(s) relate to the System (design) Specification(s).
9. The selection, implementation and use of compliant automated test tools.
10. References and Appendices, including:
    • A checklist of questions to be used when developing System Test Specification(s)
    • Templates for documenting typical system test results

In this guideline the term System Test Specification(s) refers to any of the following separate Test Specifications defined in GAMP 4:

• Hardware Test Specification
• Software Module Test Specification(s)
• Software Integration Test Specification
• Package Configuration Test Specification(s)
• System Acceptance Test Specification

Further details on the specific purpose and content of such Test Specification(s) is given later in this guideline, as well as other commonly defined testing such as Factory Acceptance Test Specifications, Site Acceptance Test Specifications and so on.

2.2 When Is This Guideline Applicable?

This guideline can be used for any project where there is a requirement for system testing and may be used to help test planning, Test Specification development, test execution, test reporting and test management.

2.3 Who Is This Guideline Intended For?

This guideline is of value to:

• Those involved with developing Validation Master Plans (VMP) and Validation Plans (VP)
• Those involved with developing Project Quality Plans (PQP)
• Those involved in reviewing and approving Test Specifications
• Those responsible for developing System (Design) Specification(s) (to ensure the ‘testability’ of the overall software design)
• Those involved with the development and execution of the System Test Specification(s)
• Project Managers whose project scope includes system testing

CHAPTER 3
Why Do We Test?

There are a number of reasons given in answer to the question ‘why do we test?’ Some of the answers are more useful than others; it is important that anyone involved in testing understands the basic reason why computer systems are tested.

3.1 Because the Regulators Require Us To…

Testing is a fundamental requirement of current best practice with regard to achieving and maintaining regulatory compliance. Although the need to test computer systems is defined by certain regulations and in supporting guidance documents, the way in which computer systems should be tested is not defined in detail.

Although the nature and extent of computer systems testing must be defined and justified on a system by system basis, it is a basic premise that most computer systems will require some degree of testing.

Failure to test will undermine any validation case and the compliant status of the system. Where exposed, during regulatory inspection, this may lead to citations and warning letters being issued and possibly a failure to grant new drug/device licenses, license suspension, products being placed on import restrictions, etc.

Regulatory expectation is based on the premise that computer systems be tested in order to confirm that user and functional requirements have been met and in order to assure data integrity. These, in turn, are driven by a regulatory need to assure patient safety and health.

3.2 Because the Quality Assurance Department Requires Us To…

The role of the Quality Assurance (QA) department (Department of Regulatory Affairs, Compliance and Validation department, etc.) in many organisations is a proactive and supportive one. In such organisations the QA department will provide independent assurance that regulations are met and will help to define policies outlining the need for, and approach to, testing.

However, in some companies this may lead to a situation where the QA department becomes responsible for policing the validation of computer systems and often defines the need to test computer systems within an organisation. The danger here is that testing is conducted purely ‘because the QA department requires it’ – other reasons for testing are not understood.

This QA role is defined at a corporate level and those organisations where the IT and Information Systems (IS) departments and QA work hand-in-hand usually conduct the most appropriate and pragmatic level of testing.

This is not always the case. In some organisations, one standard of testing may be inappropriately applied to all systems, simply because this has always been the approach in the past. It is important that computer systems validation policies state and explain the need for testing, rather than mandate an approach that must be followed, regardless of the system under test.

3.3 Because We’ve Always Done It This Way…

In many organisations there is a single standard or level of testing mandated for all.

However, one standard cannot be appropriately applied to systems that may range in scope from a global Enterprise Resource Planning (ERP) system to a small spreadsheet. In this guideline the term system covers all such systems including embedded systems. A scaleable, cost effective and risk-based approach must therefore be taken, as defined in Section 4.1.

3.4 Because It Saves Money!

So far, the only justifiable requirement for testing is based upon meeting regulatory expectation; if this were the only reason, industries not required to meet regulatory requirements would possibly not test systems at all. There is, however, an overriding reason for testing computer systems.

This primary reason for testing systems is that it is more cost effective to ‘go live’ with systems that are known to function correctly. Regulatory expectations are therefore fully in-line with business benefits.

Most people involved with projects where there has been insufficient testing know that those problems only exposed after ‘go live’ will be the most time consuming and most expensive to correct.

In many Life Science organisations there is political pressure to implement systems in unrealistic timescales and at the lowest possible capital cost. This often leads to a culture where testing is minimised in order to reduce project timescales and implementation costs.

Although this may often succeed in delivering a system, the real effect is to:

• Reduce the effectiveness and efficiency of the system at ‘go live’.
• Increase the maintenance and support costs.
• Require a costly programme of corrective actions to be implemented, to correct faults and meet the original requirements.
• At worst, roll out a system which does not meet the basic user requirements.

The net effect is to increase the overall cost of implementing the system (although this may be hidden on an operational or support budget) and to delay, or prevent the effective and efficient use of the system.

When a system is appropriately tested it is more likely to operate correctly from ‘go-live’. This improves user confidence and improves overall acceptance of the system (it is no coincidence that system or user acceptance testing is an important part of the test process). The system will operate more reliably and will cost less to maintain and support.

Senior management and project sponsors need to understand that testing is not an unnecessary burden imposed by the regulators or internal QA departments. Proper testing of the system will ensure that any potential risk to patient safety is minimised; one of the main business justifications is that it will save time and money.

CHAPTER 4
What to Test

Having stated that a ‘one-size-fits-all’ approach to system testing is no longer appropriate, the challenge is to define a justifiable approach to testing; to minimise the time and cost of testing, while still meeting regulatory expectations.

This comes down to the basic (and age-old) questions of:

• How much testing to conduct?
• What should be tested for?

Some systems are extremely complex and the concern of the regulatory agencies is that there are almost infinite numbers of paths through the software. This stems from a concern that, unless all paths through the software are tested, how can patient safety be assured under all circumstances?

In large or complex systems it is practically impossible to test each path, but the reasoning for not testing certain paths, options or functions is often made on an arbitrary basis. What is needed is an approach that will allow testing to focus on areas of highest potential risk, but to do so in a justifiable and documented manner

4.1 GxP Priority

Appendix M3 in GAMP 4 defines a methodology for determining the GxP Priority of a system. More usefully, this approach can be used to determine the GxP Priority of specific functions in a large or complex system.

In order to determine a sensible approach to testing a system it is useful to determine the GxP Priority of the system or the GxP Priority of different parts (functions) of the system. This can then be used in justifying the approach to testing. Different component parts of the system may be more GxP critical than others, for example, Quality versus Financial functions. Assessing the GxP criticality of each function allows testing to be focused on t...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• List of Tables
• List of Figures
• Author’s Preface
• Chapter 1: Purpose
• Chapter 2: Scope
• Chapter 3: Why Do We Test?
• Chapter 4: What to Test
• Chapter 5: The Test Strategy
• Chapter 6: The Development Life Cycle of a Test Specification
• Chapter 7: Recommended Content for System Test Specification(s)
• Chapter 8: Good Testing Practices
• Chapter 9: Supplier System Test Reports/Qualification Reports
• Chapter 10: The Use of Electronic Test Management and Automated Test Tools
• Chapter 11: Appendix A – Hardware Test Specification and Testing
• Chapter 12: Appendix B – Package Configuration Test Specifications and Testing
• CHAPTER 13: Appendix C – Software Module Test Specifications and Testing
• Chapter 14: Appendix D – Software Integration Test Specifications and Testing
• Chapter 15: Appendix E – System Acceptance Test SpecifIcations and Testing
• Chapter 16: Appendix F – Risk-Based Testing
• Chapter 17: Appendix G – Traceability Matrices
• Chapter 18: Appendix H – Test Script Templates
• Chapter 19: Appendix I – Checklists
• Chapter 20: Appendix J – References and Acknowledgments