9/13/2010

A Case Study y in Implementing p g an Effective CSV Program Chris Reid, Integrity Solutions Limited PO Box 71, Middlesbrough, UK TS7 0BW [email protected]

Delivering sustainable risk based solutions

1

Agenda f I. The Quality System Management support and of the functional areas Effective training and communication Self audits and customer focus

f II. The CSV Methodology Validation model and lifecycle Effective SOPs Key elements s and deliverables

f III. Spreadsheets and Queries Single-use and multi-use Verification versus validation

f IV. Roles and Responsibilities Defining the responsibilities Team concept Who is involved in the validation?

f V. How to Overcome Common Problems Audit bait Poor documentation practices Do what you say you will do Design does not meet requirements Objective evidence 2 Delivering sustainable risk based solutions

1

9/13/2010

As with anything in life life, resistance to change makes implementation of a CSV program much h more difficult 3 Delivering sustainable risk based solutions

Organisa ational Culturres

C Culture

N ti National l Cultures C lt

Projecct Culture es

R

Functional Cultures

4 Delivering sustainable risk based solutions

2

9/13/2010

Engage your “Users” in order to gain “buy in”

5 Delivering sustainable risk based solutions

Communicate Risks and Benefits

Patient Safetyy

“Real” lifecycle costs reduced

Reduced Operation and Support Cost

System Stability Compliance Reputation

Fitness for Purpose

License To operate 6 Delivering sustainable risk based solutions

3

9/13/2010

Key Principles – Risk Based Approach

• The old perception at many firms* Impact Analysis Must Be Shifted to Reflect True Risk Low Impact

Medium Impact

High Impact

• Analysis will bring us here* Impact Analysis Must Be Shifted to Reflect True Risk Low Impact

Medium Impact

High Impact

*Dictated by long-standing conservative approaches to compliance (zero risk was the ultimate goal)

*The new risk management mindset permits efficient allocation of resources to achieve an appropriate level of control that aims for low risk — not no risk

Delivering sustainable risk based solutions

Key Concepts – Risk Based Approach

Current Situation

Future Situation

High

Medium

Low

Risk Based Approach

Focus ERES / CSV Controls

Focus ERES / CSV Controls

Standard Validation Approach

High

Medium Low

Current State

Future State

Delivering sustainable risk based solutions

4

9/13/2010

Key Concepts – Validation Built on GITP / GEP Foundation Good IT / Engineering Practice Product Knowledge

Process Knowledge

Requirements

Specification and Design

Acceptance Testing and Release

Verification

Regulatory Requirements

Operation and continuous improvement

Company Quality Requirements

Risk Management Design Review Change Management

9

Delivering sustainable risk based solutions

Key Concepts – Transition of Accountability Business

Business

Who owns decision Who leads

Who reviews

IT

Who owns docs & records

Vendor

IT

Supplier

10

Delivering sustainable risk based solutions

5

9/13/2010

Key Concepts – Greater Complexity, Greater Effort

Validation Effort

Category

Description

Example

1

Infrastructure Software

Operating system, database manager, statistical programming tool, network monitoring s/w, anti-virus

2

No longer used

3

Non-Configured Products

Used ‘out-of-the-box’ with default configuration or cannot g Firmware-based be configured. appns, instruments

4

Configured Products

ERP, LIMS, CDS, EDMS, ADR

5

Custom Applications

Internally & externally developed custom applications, sometimes to modify an otherwise configured product 11

Delivering sustainable risk based solutions

Softw ware Complexity

Key Concepts - Scalability H

A D

E

B

M

L C

1

3 4 GAMP Category

F

5 12

Delivering sustainable risk based solutions

6

9/13/2010

Key Concepts - Scalability H

A

D

System Impact S

E

M C

L

F B

L

M System Complexity

H 13

Delivering sustainable risk based solutions

Key Concepts – Leverage Vendor Support

Requirements

Risk Assessment

Planning

Design

Testing

Data Migration

14 Delivering sustainable risk based solutions

7

9/13/2010

Key Concepts – Set scope

DEVELOPMENT

RESEARCH

PV

MANUFACTURING

GCP

PLANNING HIPAA FINANCE

QUALITY

GLP SOX

DISTRIBUTION

TGA

GMP

PURCHASING

GPvP

IMB

GDP

MHRA RPA FDA

SHE

GMP

MHLW

15 Delivering sustainable risk based solutions

Key Principles – Transition of Accountability

Business

Business Who owns decision Who leads

Who reviews

IT

Who owns docs & records

Vendor

IT

Supplier

16

Delivering sustainable risk based solutions

8

9/13/2010

Key Principles – Greater Complexity, Greater Effort

Category

Validation Effort

Description

Example

1

Infrastructure Software

Operating system, database manager, statistical programming tool, network monitoring s/w, anti-virus

2

No longer used

3

Non-Configured Products

Used ‘out-of-the-box’ with default configuration or cannot be configured configured. Firmware Firmware-based based appns, instruments

4

Configured Products

ERP, LIMS, CDS, EDMS, ADR

5

Custom Applications

Internally & externally developed custom applications, sometimes to modify an otherwise configured product 17

Delivering sustainable risk based solutions

Softw ware Complexity

Key Principles - Scalability

H

A D

E

B

M

L C

1

3 4 GAMP Category

F

5 18

Delivering sustainable risk based solutions

9

9/13/2010

Key Principles - Scalability

H

A

D

System Impact S

E

M C

L

F B

L

M System Complexity

H 19

Delivering sustainable risk based solutions

Key Principles – Risk Based Approach

Current Situation

Future Situation

High

Medium

Low

Risk Based Approach

Focus ERES / CSV Controls

Focus ERES / CSV Controls

Standard Validation Approach

High

Medium Low

Current State

Future State

Delivering sustainable risk based solutions

10

9/13/2010

Key Principles – Risk Based Approach

• The old perception at many firms* Impact Analysis Must Be Shifted to Reflect True Risk Low Impact

Medium Impact

High Impact

• Analysis will bring us here* Impact Analysis Must Be Shifted to Reflect True Risk Low Impact

Medium Impact

High Impact

*Dictated by long-standing conservative approaches to compliance (zero risk was the ultimate goal)

*The new risk management mindset permits efficient allocation of resources to achieve an appropriate level of control that aims for low risk — not no risk

Delivering sustainable risk based solutions

Key Principles – Leverage Vendor / Project Team Support

Requirements

Risk Assessment

Planning

Design

Testing

Data Migration

22 Delivering sustainable risk based solutions

11

9/13/2010

Laws and Regulation, Corporate Policies and Standards

Business Strategy and Requirements, IT Strategy

Policies Measure

Accountabilities & Capabilities Learn

Processes Service Desk D

Docume ent Managem ment

Training Re ecords Managem ment

Other systtems

Standard Operating Procedures

Improve

Work Instructions Documentation and Records Delivering sustainable risk based solutions

What is a Computerized System?

SOFTWARE

HARDWARE COMPUTER SYSTEM

OPERATING PROCEDURES EQUIPMENT (OPTIONAL) CONTROLLED FUNCTION

COMPUTERIZED SYSTEM OPERATING ENVIRONMENT

Delivering sustainable risk based solutions

12

9/13/2010

What is a Computerized System?

25 Delivering sustainable risk based solutions

Key Concepts – Cradle to Grave

RECOMMENDATION*

Retirement” PROJECT INITIATION* O Aging”

P E R

RELEASE FOR USE*

A T

M t it ” Maturity” I O

* = Event “ = Period

N A

E

L L

I

F 26

Delivering sustainable risk based solutions

13

9/13/2010

Lifecycles User Requirements Specification

Functional Specification

Functional Testing

Design Specification

Specification

Primary  Responsibility Regulated  Company

Requirements Testing

Module (Unit) Specification

Integration Testing

Verification

Module (Unit) Testing

Code Modules

Supplier Supplier QMS

27 Delivering sustainable risk based solutions

G1

Integrated Lifecycles

Idea, Feasibility & Business Case

Initial Project Definition

Proposed Business Benefits, Business / IT Strategic Fit, Resources, Milestone Plan, Potential Project Risks

Project Stages

Regulatory + GPP

Initial Planning

Budget resource and technology costs Return on Investment, Initial Project Plans

Good Project Practice

Typical Vendor Activity

Initial Risk Assessment

G2

Portfolio Management

G3 Project Definition & Organisation

Portfolio Review

G5

Detailed Planning

Business/Manufacturing Processes

Business Process Risk Assessment

Criticality of Business / Manufacturing Processes

Solution Evaluation

Product evaluation, Supplier demonstrations q fit,, Validation implications p Technical fit,, Requirements Vendor Proposals / Tenders, Business , Quality and Commercial Evaluation and Selection of Solution

Vendor Assessment

Quality Organisation, Systems and Application Technical Capability, Experience & Resource Availability, Support Capability

Project Plan

W ork breakdown, Scope, Structure, Timescale plan, Critical path analysis, Resources

Contract

Terms and Conditions Phasing, Organisation, Fixed Price, Reimbursable

Validation Plan

Validation Roles, Approach, Controls, Deliverables DJO vs. Vendor Activities and Deliverables Cost Breakdown, Investment Phasing Stage Payments

Servers, Data Storage Operating System, Security/Backup

ERES Risk Assessment Detailed Software Spec Requirements Traceability

G7

Build Solution

Code & Configure Development Testing

G8

G9

Acceptance

Go Live

Test Planning

Hardware Specification

Unit/Module Specifications Software structure and logic

Project Planning and Monitoring

Configuration, fields, screens, reports Customisations, new functionality, database schema

Infrastructure Specification

Internal Audit

Process flow, Functionality, Reports, Data Input Processing, Communications, Exception Handling, Database schema, etc

Functional Risk Assessment

Change Management

Functional Specification Configuration Specification

Document Management

Design & Design Review

Risk and Deviation Management

Design

Definition and Analysis of User, Functional and Technical Requirements

Request for Proposal

Investment & Cost Plan

G6

Importance vs. other project commitments, Capacity to deliver, commitment to resources Stakeholder commitment, Project Go / No Go

Business Process Mapping

Requirements Management

G4 Solution Evaluation

Initial Assessment of Project Risks, Regulatory Risks and Project Commitment

System Construction, Interfaces Hardware Configuration, Cabling/wiring

Coding Standards, Code Reviews Structural Testing, Integration Testing Custom code Test Cases, Objectives, Phasing, Relationships, Test Failure Management, Technical and Human Resources

IQ, OQ, PQ Interim Reports

Installation verification, Challenge Testing Functional Testing, Performance / Process Testing

Data Migration/ Data Input

Data Cleansing, Migration Planning, Migration Testing

Training Production IQ

Handover

G10

Project, System / Audit & Review

Operate

G12

Retire

Handover to Operational and Support Organisations

Conformance to Validation Plan

Post Implementation Monitoring

Process and System Effectiveness and Stability

Project Learning Review

G11

Hardware & Software Installation

Validation Report

Planned vs. Actual, Benefits, Plan, Resource, Contract Strategy Quality Strategy, Requirements

Change Control, Configuration Management, Security, Backup and Restoration, Disaster Recovery, Business Continuity, Maintenance, Periodic Review Software Archive Records Archive & Retention

28

Delivering sustainable risk based solutions

14

9/13/2010

Vendor Assessment

Initial determination of project risks, regulatory risks (GxP, SOX, Data Privacy). Determines whether the application or infrastructure has any impact or not (but not scope of impact) Need for, and rigor of, vendor assessment

Validation Planning

Role of vendor based on outcome of assessment e.g. use of vendor documentation, use of vendor’s input into activities Determining activities, deliverables and responsibilities

Plannin ng Risk Assessm ment Activitties and Objectiv ves

Initial Risk Assessment

Business Process Risk Assessment User Requirements

Planning Risk Assessment and Management activities Identify criticality of business processes, adequacy of process definition and controls Identify Business, Safety/Health/Environment and Regulatory requirements

Functional Risk Assessment Identify criticality of functional design, adequacy of design and controls requirements Electronic Records Risk Assessment Requirements Traceability

Identify criticality of electronic records and associated electronic signatures maintained by the computerized system Determines deviations between requirements, design and testing

Test Planning

Scope and phasing of testing based on Process and Functional Risk Assessments Use of Vendor Testing Test environments Test documentation Assessment of deviations from Validation Plan and Lifecycle Activities System availability

Validation Reporting Operational Planning

Frequency and level of backup and recovery Disaster recovery planning System Security Change Control and impact assessment Frequency of Periodic Reviews and Audit

29

Deviation management and CAPA

Delivering sustainable risk based solutions

Risk Management Log Step1: Risk Identification

Step 2: Risk Scenario

Step 3: Risk Step 4: Type Severity of Impact

Step 5: Likelihood of Occurrence

Step 6: Risk Step 7: Risk Class Detection

Step 8: Risk Step 9: Risk Step 10: Priority Controls Assignment

Name Date

Insufficient Validation Resources

System not adequately tested

Business and Compliance

High

High

1

High

Medium

Recruit additional resources

CR

12/1 0/09

30 Delivering sustainable risk based solutions

15

9/13/2010

Risk Determination

Low

Medium

High

High

2

1

1

Medium

3

2

1

Level ONE

Level TWO Probability of Detection

3

High

3

Medium

Low

Low

Level THREE

H

H

M

2

H

M

L

3

M

L

L

2

Risk Classification n

Severity of Impact

Risk Likelihood

1

HIGH priority

MEDIUM priority

LOW priority

31 Delivering sustainable risk based solutions

System Inventory Field System Name

Purpose / Description Descriptive Name of System e.g. Document Management System

System Reference System Description

Unique System Identification Brief Description of System Use (e.g. business processes supported)

Computerized System Version

Version of main application / system software component

Location Business System Owner (Business Process Owners for integrated systems)

Location where application installed / deployed. Name of System Owner (may be multiple for business applications)

Technical System Owner

Name of the Technical System Owner (normally IT Manager)

g Status Regulated

GxP,, HIPPA,, SOX,, non regulated, g , etc

Primary GAMP Category*

GAMP software category 1 – 5

Validation Status*

Not Required, Not Validated, Ongoing, Validated (including date of Validation Report sign off)

Electronic Records & Electronic Signatures

Indication of whether system holds regulated electronic regulated records

Date of last Periodic Review* Periodic Review frequency* Date of next Periodic Review*

Date when last period review report approved 1, 3 or 5 years Date of next periodic review (last review date + frequency)

32

Delivering sustainable risk based solutions

16

9/13/2010

Documentation Considerations

Supplier documentation

Development and Acceptance test documents Functional and detailed design g

33 Delivering sustainable risk based solutions

IM MPACT RESULTS

Initial Risk Assessment – Project Risk Summary IS THE PROJECT SIZE ACCEPTABLE AND MANAGABLE?

YES 

NO

IS THE PROJECT DEFINITION CLEAR AND ACCEPTABLE?

YES 

NO

IS PROJECT SPONSORSHIP AND COMMITMENT CLEAR AND ACCEPTABLE?

YES 

NO

IS THE IMPACT ON USER ORGANIZATION CLEAR AND ACCEPTABLE?

YES 

NO

IS THE PROPOSED TECHNOLOGY MANAGEBLE WITH CURRENT SKILLS  SET?

YES YES 

NO

IS THERE AN ADEQUATE PROJECT TEAM THAT IS AVAILABLE TO FULFILL  DEFINED ROLES?

YES 

NO

IS THE PROJECT MANAGER EXPERIENCED, SKILLED AND ABLE TO MEET  PROJECT DEMANDS?

YES 

NO

34 Delivering sustainable risk based solutions

17

9/13/2010

IM MPACT RESULTS

Initial Risk Assessment – Regulatory Impact Summary

IS THE SYSTEM SUBJECT TO ANY GLOBAL HEALTHCARE REGULATIONS?

YES  NO

IS THE SYSTEM SOX / PCI / HIPAA REGULATED?

YES  NO

DOES THE SYSTEM HOLD, CREATE, MODIFY, DELETE OR DISTRIBUTE REGULATED  ELECTRONIC RECORDS?

YES  NO

IF THE SYSTEM IS A LEGACY SYSTEM AND MANAGES HEALTHCARE REGULATED  ELECTRONIC RECORDS HAS AN ELECTRONIC RECORDS AND ELECTRONIC ELECTRONIC RECORDS, HAS AN ELECTRONIC RECORDS AND ELECTRONIC  SIGNATURE ASSESSMENT BEEN CONDUCTED?

YES  YES NO

DOES THE SYSTEM EXECUTE ELECTRONIC SIGNATURES TO REGULATED  ELECTRONIC RECORDS

YES  NO

35 Delivering sustainable risk based solutions

Validation Planning Vendor Risks

Requirements Criticality Size and Complexity

Validation Approach

Leverage Supplier Work

Standard vs Bespoke

In-source / Outsource

Innovative or Routine

Team Org Org, Roles & Responsibilities

Single Site/ Multiple Sites

Documentation Requirements, Org, Ownership

Resource Skills And Availability

V. Plan Roles and Resp Approach Deliverables Etc Etc

36 Delivering sustainable risk based solutions

18

9/13/2010

Risk Area

Determined from

Validation Strategy Decisions

Criticality of the Computerized System

Business Process Risk Assessment

Degree of effort required in documentation and testing

Requirements categorization Electronic Records and Electronic Signatures Risk Assessment

Need to engage Business QA in the project Need and approach to vendor assessment

Resource availability and skills

Initial Risk Assessment

Need for in project audits Need to procure additional / specialist resources

System Complexity:

Validation and Project Planning

Need to phase project System Development / Implementation Lifecycle

•

Level of configuration required

•

Level of customization required

•

Size

•

Number of users

•

Interfaces to other systems

Expanded or condensed documentation set Need for design reviews and functional risk assessments N d ffor iin project Need j t audits dit Roll out planning

• Novelty of solution Vendor Capability

Vendor Assessment

Legacy system implications

Validation and Project Planning

Effort that can be leveraged from vendor Documentation that can be leveraged from vendor Need for System Retirement Planning and Data Migration 37 Planning

Delivering sustainable risk based solutions

Validation Planning f Validation Plans define: System Implementation and site quality assurance approach Organization, Roles and Responsibilities SOPs governing lifecycle activities Documentation and other Deliverables Risks and Risk Management Plans Relationships with vendor activities and documentation System / Project Acceptance criteria

38 Delivering sustainable risk based solutions

19

9/13/2010

Validation Planning Infrastructure Software

Layered software (i.e., upon which systems are built) Software used to manage the operating environment

Operating Systems, Database Engines Middleware, Programming languages Statistical packages, Spreadsheets Network Monitoring tools Scheduling tools, Version control software

Qualification Plan Configuration Specification Installation Qualification Qualification Report (See Risk Based Approach to Implementation, Qualification Support Qualification, Support, Retirement of IT Infrastructure Infrastructure, CSC.POL.0003,

Non-Configured Run-time parameters may be entered and stored, but the software cannot be configured to suit the business process

Firmware-based systems COTS software Instruments

Validation Plan Requirements Specification Functional Risk Assessment Electronic Records and Signatures Risk Assessment Installation Qualification Combined Installation Qualification / Performance Qualification Validation Report

Configured

Software, often very complex, that can be configured by the user to meet the specific needs of the user’s business process. Software code is not altered

LIMS, Data Acquisition Systems SCADA, ERP, MRPII, Clinical Trial Monitoring DCS, ADR Reporting CDS, EDMS, Building Management Systems CRM, Spreadsheets Simple Human Machine Interfaces (Note: Specific examples of some of the above may contain significant custom developments)

Business Process Maps Business Process Risk Assessment Requirements Specifications Vendor Assessment Validation Plan Configuration Specifications Hardware or Infrastructure Design Specification Functional Risk Assessment Electronic Records and Signatures Risk Assessment Requirements Traceability Matrix Installation Qualification (Test Environment) Operational Qualification Performance Qualification Installation Qualification (Production Environment) Validation Summary Report

Custom

Software custom designed and coded to meet business process

Varies, but includes: Internally and externally developed IT systems Internally and externally developed process control systems, Custom ladder logic Custom firmware, Spreadsheet macros

Same as configurable plus: Functional Specifications Software Design Specifications (if needed) Software Code Review (critical code) Development Testing

39

Delivering sustainable risk based solutions

Document Management Plan Attachment 1 - Validation Document Set Any document number not yet assigned (NYA) will be documented in the Final Validation Report. Documents identified in the table as “A” are  applicable for all Waves and sites Documents identified in the table as “B” applicable for all Waves and sites.  Documents identified in the table as  B  are only applicable to a particular Wave or site.  are only applicable to a particular Wave or site Future  Wav Document  Document  Doc # Wave( Document Author Document e 1 Reviewers Approvers s)  System A A Validation  Global IT  System Owner(s)  Validation Plan (SVP) Specialist(s) Global IT Governance and  Quality System Owner Compliance  Global Software Validation   Document  Management Plan (DMP)

A

A

Validation  Specialist(s)

Global IT Governance and  Compliance 

Validation Project Manager  System Owner(s) Quality System Owner Global Software Validation 

 System  Description (SD)

A

A

Validation  Specialist(s)

Global IT Governance and  Compliance  Global IT  Implementation Team

Vendor Audit Report(s)

A

A

Validation  Specialist(s)

Global IT Governance and  Compliance  Global IT 

Validation Project Manager  Validation Project Manager System Owner(s)  Quality System Owner  Global Software Validation  Validation Project Manager  System Owner(s)  Quality System Owner  Global Software Validation  Validation Project Manager  40

Delivering sustainable risk based solutions

20

9/13/2010

Vendor Assessment Vendor A

Vendor B

Vendor C

QUALITY ORGANIZATION QUALITY SYSTEM EXISTS QUALITY SYSTEM APPLIED QUALITY SYSTEM MAINTAINED

⌧ ⌧ ⌧

TECHNICAL COMPETENCE

⌧ ⌧

PHARMA EXPERIENCE SUPPORT INFRASTRUCTURE COMMERCIALLY ROBUST

⌧ ⌧ ⌧ ⌧ 41

Delivering sustainable risk based solutions

Vendor Assessment Method

Typical Criteria for Method

Postal Audit Non patient safety related processes Questionnaire / Desktop Identifying areas to focus on during site audits Audit Filtering vendors during product and vendor

selection processes Follow-up of observations from previous audits Surveillance audits Site Audit

Patient Critical Systems Complex and / or Custom Systems

Performance Review

Interview

Existing vendors / service providers where experience of actual working practices and experience is at hand Where individuals are supplied from a company to work to internal procedures.

42

Delivering sustainable risk based solutions

21

9/13/2010

Vendor Assessment f Planning Considerations: Scope and criticality of system / service use Whether a managed solution / service is to be provided or whether external resources shall work to internal’s procedures under internal management The extent to which the system / service is used in industry and evidence that problems raised by customers are addressed Whether there has been a significant change in scope of systems / services provided or whether there have been significant changes in regulatory requirements since previous assessments Is there confidence in the Vendor based on previous experience? System maturity Complexity of system (e.g. will validation itself fully prove the system is fit for intended use) Is the system based on new and relatively unproven technologies? 43 Delivering sustainable risk based solutions

Vendor Assessment

Background and Audit Purpose Details of Audit

Deficiencies discovered and corrected during the audit

Name of Organization being audited

Confidential information without express permission of organization

Date of Audit Audit Location Audit Team Vendor Team Summary of Observations & Risks Detailed Explanation of Findings (strengths and weaknesses) Recommendations for selection, rejection, corrective action and follow up Identification of responsibilities for actions identified.

Subjective opinion or ambiguous statements Antagonistic words or phrases 44

Delivering sustainable risk based solutions

22

9/13/2010

Vendor Assessment f Observations Critical: A finding that would disqualify a facility, a study or portion thereof or significantly jeopardize patient safety or accuracy of reported regulatory data Major: Non-compliance against internal policies, external regulations, internal protocols, Standard Operating Procedure, Statement of Work or other contractual agreement or obligation Minor: A finding which may not affect operations or conclusion of a study or does not meet the current acceptable standards or practice.

45 Delivering sustainable risk based solutions

Business Processes f Business processes: Id tif main Identify i process steps Highlight computerized system interactions with the process Identify process interfaces Identify critical data flows Identify process inputs and outputs Identify manual interventions

Data Movement

46 Delivering sustainable risk based solutions

23

9/13/2010

Business Process Risk Assessment Process Step

Function

Sub Functi on

QU003 – QA Release (including CoA Generation) E Enter R Results l iinto QMS N/A N/A

Risk

Risk Scenarios

Impact (H, M, L)

Detecti on (H, M, L)

Priority

GXP

I Incorrect results l entered

H

M

H

N/A

N/A

GXP

Enter results against wrong test / item

H

M

H

Step 1: QMS updates lot status

N/A

N/A

GXP

H

M

H

Step 3: Create and Print CoA by authorised person

N/A

N/A

GXP

Incorrect lot status is assigned based on comparison of results to specification CoA not printed

H

H

M

N/A

N/A

GXP

Steps 2 and 5: QP changes the lot status to approved or reject depending on test outcome

N/A

N/A

GXP

N/A

N/A

GXP

N/A

N/A

GXP

QP signs CoA (as printed from the system) Step 4: Warehouse transfer to write off materials

CoA contains incorrect information Wrong status is assigned

Control / Corrective Action Measures

(H, M, L)

H

M

H

H

M

H

Approved status is assigned from a failed test status

H

M

H

Unauthorised person approves material

H

M

H

N/A

N/A

GXP

N/A

N/A

N/A

N/A

N/A

N/A

GXP

Inventory Transfer does not adjust inventory and location

H

M

H

A ti Actions: SOP: Verifies that correct results have been entered (PRA SOP Ref 6.9.3) Actions: Test: Ensure that results cannot be entered against wrong item (i.e. invalid test spec for item) (PRA Test Ref 6.9.3) Actions: Test: Confirm that results out side of each defined test limit result in failed test status (PRA Test Ref 6.9.4) Actions: Test: CoA printed and contains correct information (PRA Test Ref 6.9.5)

Actions: Test: Ensure that the correct status is assigned, as entered by the QP. (PRA Test Ref 6.9.6) SOP: Provides instruction regarding entry of correct status. (PRA SOP Ref 6.9.4) Actions: Test: Confirm that approved status cannot be assigned if the test failed (PRA Test Ref 6.9.7) Actions: Ensure only QP can approve material (PRA Test Ref 6.9.8) Actions: Manual sign off of the CoA Actions: Test: Ensure that inventories and locations for manufacturing and

47

Delivering sustainable risk based solutions

Requirements Traceable

Req. No. X.0 x.1 x1 1 x.1.1

Ranked (Importance) x.1.2

x.1.3

Testable

x.1.4

x.1.5

Unambiguous

Grouped by Function (Theme)

x.2 x.2.1 x.2.2 x.2.3 x.2.4

Description Security Requirements Access Control/Security Hierarchy

Ranking

f Maintained

Syste secu System security ty must ust allow a o for o tthe e following o o g use user types types: a.) System Administrator b.) Production Supervisor c.) Production Operator d.) Data Reviewer

RC

System must provide System Administrator with authorities as defined in the Security Hierarchy Matrix. (Appendix x) System must provide Production Supervisor with authorities as defined in the Security Hierarchy Matrix. (Appendix x) System must provide Production Operator with authorities as defined in the Security Hierarchy Matrix. (Appendix x) System must provide Data Reviewer with authorities as defined in the Security Hierarchy Matrix. (Appendix x) Password Expiration Passwords must expire every 90 days Upon expiration of the password, the user must be prompted to reassign a new password. The user must be granted three grace log-ins after initial expiry notification. The system must not allow the reassignment of previously used passwords. Delivering sustainable risk based solutions

RC

RC

throughout the project

f Need not be maintained post go-live

RC

RC

RC RC RC RC

f New N Specification created for significant changes post go-live

24

9/13/2010

Roles Aspect

Purpose

Process Knowledge

In order to identify key requirements of the system that are related to the business or manufacturing process

Business Knowledge

To ensure that requirements are challenged against business need and are realised

Ownership

To ensure clarity and understanding of stated requirements

Analytical

To ensure requirements are challenged to ensure they are accurate and complete

Technical/Product

To ensure that requirements are practical in terms of available technology

Process/Product Impact

To ensure requirements which impact the process or product are clearly identified

Technical Authorship

To ensure that requirements are written in concise, correct, and unambiguous language Delivering sustainable risk based solutions

Requirements Specification Content f Requirements Specifications shall be established in order to amplify business Process Steps and to define user, functional and technical requirements. Typically, Requirements Specifications shall address: Process requirements Process limits User interface System interface requirements Data management Electronic Records and Electronic Signatures requirements Security controls Health, Safety, Environmental requirements Equipment needs Operating context / environment Regulations, internal and external standards to be met Future development needs 50 Delivering sustainable risk based solutions

25

9/13/2010

Avoid Requirements that cannot be tested

Provide easy navigation between screens Provide periodic auto-archival of data Allow deletion of data when applicable Provide appropriate security levels Generate the required reports

Delivering sustainable risk based solutions

Design and Design Review Business Processes

Requirements

Standard System Manuals (COTS)

Functional F i l Specification

Software Specifications

Configuration Specifications

Hardware / Infrastructure Design

52

Delivering sustainable risk based solutions

26

9/13/2010

Infrastructure Design f SYSTEM ARCHITECTURE AND Infrastructure applications  and processes

• Enterprise Resource Planning • Laboratory Information • Adverse Event Reporting • Advance Planning • HR • etc

• Configuration Management • Service Desk • Security Management • Virus Management • Backup and Restoration • Problem Management • Network Monitoring • etc

Infrastructure Systems

Business Systems

Business applications and  processes

Shared Platform

INTERFACES SERVICE DEFINITION CABINET LAYOUT PERFORMANCE DATA CENTERS DATA STORAGE SECURITY NETWORK CONFIGURATION CLIENT CONFIGURATION SERVERS POWER AND ELECTRICAL SUPPLIES f ADMINISTRATOR INTERFACE f INTERFACES AND COMMUNICATION PROTOCOLS f REMOTE ACCESS

f f f f f f f f f f

Clients IT Services (e.g. DBMS, F&P, Email) Operating Systems Server Hardware Network Environment

IT Infrastructure

Delivering sustainable risk based solutions

Functional Risk Assessment - BMS FS Section 3.1.1

3.1.2

Function TE04 – Inlet air temperature monitor

Sub Function

N/A

Inlet Flow Control N/A

NA

3.12

RISK Risk Scenarios TE04 fails, no impact as this is only for None visual monitoring of external air temperature Fail open, required closed Heating and cooling B units would freeze in winter if HVAC shutdown but FV01 remained open Fail closed, required open If FV01 closed and HVAC is running a B potential “vacuum” would be created and equipment / ducting damage may occur

Impact

Detect ion

Priority

N/A

N/A

N/A

M

H

L

M

H

L

Mitigation Measures

1.

None

1.

Temperature alarm TZ01 would indicate that fluid is –ve temperature Interlock closes FV01 when fan TIK2 not running – Verify that the interlock has been tested

2.

1. 2.

Pre-filtration

3.

1.

2. NA

B

Pre-filter blocks

L

H

L 3. 4.

Visibly damage the equipment Low pressure alarm on air inlet to controlled area Interlock prevents fan TIK2 running if FV01 closed – Verify that the interlock has been tested PDE01 would detect an out of range differential pressure across the filter and alarm – Verify that the alarm has been tested. PDE01 calibrated annually (SOP ) Pre-filters changed annually (SOP ) Routine weekly check to ensure differential pressure is within limits 54 (SOP )

Delivering sustainable risk based solutions

27

9/13/2010

Electronic Record Risk Record Criticality

Vulnerability to unauthorized change

Risk

Ability to detect changes

55 Delivering sustainable risk based solutions

Risk Vulnerability (Probability)

Low

Medium

High

Calculating g Record Risk Medium Impact

Low Impact Probability of Detection

Low

Low

Medium

Medium

High

High High

Medium Risk Category

Electronic Record Imp pact

High Impact

Low

High

Medium

Low

Delivering sustainable risk based solutions

28

9/13/2010

Controls based on criticality of record High Impact:

Medium Impact:

Records that may directly affect either:

E.g. but not limited to:

•

Patient safety

Bill of Materials

•

Product quality quality, safety safety, identity identity, or efficacy

Design History File

•

Data integrity for regulatory submission

Recall Records

•

Distribution and recall

Complaints Records

•

Regulated Financial Reporting

Financial Records

•

Confidential Healthcare records

Records that indirectly affect either:

E.g. but not limited to:

•

Patient safety

•

Product quality, safety, identity, or efficacy

•

Data integrity for regulatory submission

Validation Documentation

•

Distribution and recall

SOPs

Training Records

Audit and Investigation Records Low Impact

Records that have no impact on:

E.g. but not limited to:

•

Patient safety

Planning Records

•

Product quality, safety, identity, or efficacy

Progress Monitoring Records

•

Data integrity for regulatory submission

•

Distribution and recall Delivering sustainable risk based solutions

Record Impact

Required Controls

Recommended Controls

High

Unique user accounts 2 component or biometric security code / e e-signatures signatures Electronic Audit trail Password Ageing Automatic user session timeout following inactivity Manual user session lockout

Medium

Unique user accounts 2 component or biometric security code / e-signatures Manual user session lockout Audit trail of system changes (e.g. change control records)

Electronic Audit trail Password Ageing Automatic user session timeout following inactivity

Low

Password or Network Security Audit trail of system changes (e.g. change control records)

Unique user accounts Automatic user session timeout following inactivity Manual user session lockout

58 Delivering sustainable risk based solutions

29

9/13/2010

Configuration Management Identify configuration item Hardware Software Documentation, SOPs Configuration item control Effective change control Change traceability Backup, Version control

Configuration Item (Software, Hardware, Documentation, SOPs)

Status accounting Status of all configuration items is known

Configuration verification Build Controls, Installation and Operational Qualification

59

Delivering sustainable risk based solutions

Configuration Management f Configuration Management is required in support of: Change management Problem investigation Disaster recovery in the event of system failure

f Configuration management shall ensure: Business decisions leading to configuration settings are recorded Ch Changes are recorded d d All configuration parameters can be recovered from a secure record (electronic backup or paper)

60 Delivering sustainable risk based solutions

30

9/13/2010

Configuration Management f A configuration baseline shall be established at system go live in order to support problem investigation

f Configuration Specifications, log books or other suitable form shall record configuration changes post go live

f All configuration changes shall be traceable to the system release in which the change was made

f Electronic configuration files shall be subject to version control (where possible) and controlled backup. Security controls shall minimize the risk of unauthorized or inadvertent change

61 Delivering sustainable risk based solutions

Software Coding Standards and Review Standards

Availability y of, and conformance to coding g standards

Structure

Modular, Module Size, Control Blocks

Readability

Layout, Naming Conventions, Annotation/Comments

Data

Local vs Global, Parameter Passing (mechanism)

Scope

Functions stated in design implemented

Logic

Critical functions e.g. calculations

Integrity

Deadcode, redundant code

Control

Configuration Management, Versioning, Change Control traceability 62 Delivering sustainable risk based solutions

31

9/13/2010

Development Environments

Training

Sandpit

(Subject to GITP, used for training system users)

(Not controls, used for investigation and trials)

Copy

Copy

Development (Subject to GITP, used for code creation, configuration & Dev Testing)

Test / Validation Version Control Code Review Successful Dev Test

(Subject to IQ, used for OQ and PQ)

Production Successful OQ / PQ Release Management First Release version 1

(Subject to IQ, used for live PQ and Operation)

63 Delivering sustainable risk based solutions

Principles of Risk-based Qualification ‘S ‘Supplier’s li ’ standard t d d inspection i ti and d test t t documentation d t ti may be used and no other documents be produced that duplicate this information, provided that documentation clearly shows the items of interest have been verified or tested in an appropriate manner. This is subject to the supplier being of adequate quality.’

Delivering sustainable risk based solutions

32

9/13/2010

Test Planning f A Test Plan shall be created for all computerized systems systems. The Test Plan shall define: List of tests to be conducted (positive and challenge tests) Test Objective Expected Result against Test Objective Relationships / dependencies between tests Equipment required to conduct tests Data required to conduct tests Test resources Test failure management procedures

65 Delivering sustainable risk based solutions

SOPs Supporting Testing f SOPs / Work Instructions should be used in support of testing Ensures consistency of testing Verifies SOPs / Work instructions

f SOPs / Work Instructions used in support of testing must be managed: Ensure current issue of SOP / Work Instruction being used Ensure SOPs / Work Instructions used in testing are written before testing Ensure that errors in SOPs / Work Instructions are addressed

66 Delivering sustainable risk based solutions

33

9/13/2010

Development Testing f Development testing shall be conducted for any custom developed software by the system developer prior to qualification. Development testing shall challenge the structure and logic of the developed code.

f Development testing shall be conducted in accordance with the same controls as qualification testing however development testing does not need to be independently reviewed by Site Quality Assurance Assurance.

f Development test records should be retained and made available for periodic Site Quality Assurance audit.

67 Delivering sustainable risk based solutions

Installation Qualification Documented verification of the installation f Installation verification shall and configuration of test and production confirm correct installation of: environment hardware and software. Hardware Operating system and other Installation shall be controlled by written infrastructure software procedures and / or protocols. Application software System configuration Security setup

f IQ of IT Infrastructure addressed by CSC.POL.0003, Risk Based Approach to Implementation, Qualification, Support and Retirement of IT Infrastructure. 68 Delivering sustainable risk based solutions

34

9/13/2010

Operational Qualification f The Operational Qualification Protocol shall cover: Functional/Operational tests vs. design Challenge testing against operating ranges (e.g. data entry, performance) Communication interfaces Start-up & shutdown Security and access Data storage and recovery Alarm, event status handling

Delivering sustainable risk based solutions

Performance Qualification

f PQ comprises i product d t performance and / or process performance qualification (vs URS)

f Performance P f related l t d functionality / processes

f System Interface Performance f Management controls – Monitoring of user enquiries Data changes System availability System stability Problem resolution to Incident Logging – System changes

– – – –

Delivering sustainable risk based solutions

35

9/13/2010

Acceptance Criteria

Traceable

Range

Accurate

Maintainable

Sensible

Related to critical process parameters Predefined “Worst case”

Delivering sustainable risk based solutions

Results Recording f Results must clearly demonstrate that acceptance criteria has been met f Be positive / specific For values record actual value – 20.3 deg C NOT > 20 deg C

For other items e.g. messages – As acceptance criteria

f No tick boxes, ok, etc

Delivering sustainable risk based solutions

36

9/13/2010

Expect the Unexpected!

• It is inevitable that changes g will be required during testing • Changes must be managed – No quick fixes – Don’t pressure developers – Consider need for regression testing

– Use run number to identify repeat tests

– Changes must be managed Delivering sustainable risk based solutions

Objective Evidence f When ever possible generate objective evidence in support of test results printouts screen dumps logs photographs certificates charts

Delivering sustainable risk based solutions

37

9/13/2010

Test Summary Report f Test Summaryy Report p shall: Provide a status of all testing conducted Provide a list of test incidents Provide a summary of actions taken to address test incidents Identify the number of test runs conducted for each test

f Depending on the phasing and scope of testing, a test summary report may be created after each test phase, at the end of testing or in the overall project Validation summary Report Report.

Delivering sustainable risk based solutions

Leveraging the Supplier’s Documentation f Supplier Supplier’s s documentation Should be assessed for suitability, accuracy and completeness Allowed flexibility regarding acceptable format, structure and documentation practices

f Consider Style Level of Approvals (e.g. Site QA) Management of Deviations

f Quality of supplier documentation f Need for oversight of testing f How supplier testing will be integrated with end user testing

Delivering sustainable risk based solutions

38

9/13/2010

Test Case Header Test Case Number

OQ-1.0

Test Case Name

Test Case Objective Test Case Acceptance Criteria

Pass Criteria - Actual result recorded is same as the expected result of the test Fail Criteria - Actual result is not same as the expected result

Test Case Set-up (Equipment, Data, Related Tests) Test Case Comments Test Case Start Date/Time

Test Case Finish Date/Time

(dd-mmm-yyyy /hh:mm)

Pass / Fail

(dd-mmm-yyyy /hh:mm)

Test Case Executed By: (Initial and Date)

Test Case and Related Results Reviewed By (Initial and Date)

Delivering sustainable risk based solutions

Test Case Body OQ-1.1

Step No.

SubTest

Step Description

Expected Result

Actual Results Incident Number Pass / Fail (if any) Comme nts

Actual Result Evidence Reference

Initial and Date

OQ-1.1.1 OQ-1.1.2

Delivering sustainable risk based solutions

39

9/13/2010

Test Incidents

Incident Number:

IN.003 System Name and Id

Test Reference (where relevant) Title: Test

Global ERP

OQ.002 BOM Printing, failed at step 5

Person Reporting Incident: Chris Reid Person Managing Incident: A.N. Other Date Reported:

15th July 2009

Closed By:

A.N. Other

Department: IT Department: Development Impact (Business / GxP) Date Closed:

GxP 21st July 2009

79 Delivering sustainable risk based solutions

Test Incidents Description of Incident (Describe Incident, how did it manifest itself, what actions were taken prior to the incident occurring)

Failed to meet acceptance criteria set forward in test OQ.002, step 5

Incident Impact (What was/is the impact of the incident, service disruption, data impact, current availability, validation testing, etc. Is there any impact on business or regulatory critical applications, services or data)

Test OQ.001 needs to be re-tested Data supporting test OQ.002 requires reset to original values

Investigation and Diagnosis (Outcome of investigations into cause and effect)

Error in software module “Print BOM”

Action Taken (What actions have been taken to remedy the incident and what actions have been taken to minimize the risk of recurrence in the future)

Change control CC.0032 raised to address software error and test OQ.001 and OQ.002 re-executed successfully

80 Delivering sustainable risk based solutions

40

9/13/2010

Validation Summary Reporting Sections of Validation Plan

Sections of Validation Report User Requirements Specification

Introduction (Description)

Introduction

Scope

Scope Supplier Audit

Validation Report

Organisational Structure

Phase by Phase Project-specific Validation Plan

GxP Criticality Assessment Validation Strategy; DR/IQ/QO/PQ Change Control

PQ

Functional Specification

Summary of Results Details of Execution

OQ

Deviation Reporting & Resolution

Design Review and Risk Assessment Design D i Specification

IQ

Validation Status

SOP’s and Training

Confirmation of Training

System Build

Documentation Management

Glossary Responsibilities and Resources Glossary

Iterative Process

Information Flow

Process Flow

Appendices

81 Delivering sustainable risk based solutions

Go Live f Ensures operations and support organizations are prepared to receive the validated system: » Documentation updated to as built status » Operation and support SOPs in place » Technical and support documentation available to support staff

» Internal and external SLA are in place » Operations and support personnel trained in system use and SOPs

» Residual risks from validation have been evaluated and actions defined.

» Operations take ownership of risk mitigation » Support organizations trained

82 Delivering sustainable risk based solutions

41

9/13/2010

Post Implementation Monitoring f System stability and availability shall be monitored f a period, for i d post go live: li

RECOMMENDATION*

Extent of changes post go live Number of workarounds implemented Number of issues / incidents raised and their severity System failures System availability against Service Level Agreements System and Infrastructure performance

Retirement” PROJECT INITIATION* O Aging”

P E

f Focused monitoring and regular reporting shall

R

RELEASE FOR USE*

A T

Maturity” I

continue until full business as usual status is achieved hi d and d statistics t ti ti d demonstrate t t stable t bl and d acceptable operation.

f Report created to summarize the outcome.

O N A

E

L L

I

F 83 Delivering sustainable risk based solutions

Operational Controls

D Develop l

Planning

O Operate, t Maintain M i t i

Performance Monitoring

Reporting

Specification

Verification

Configuration and or coding

Service Management Handover Periodic Review

Internal Audit

System Inventories Change Management Configuration Management Repair Training

Document Sys Admin BCP & DRP Records Management Management Backup & Restore

R ti Retire

Document & Software Archive Record Archive Record Retention Data Migration 84

Delivering sustainable risk based solutions

42

9/13/2010

Change Request and Evaluation

Change Implementation Register Change

Change Request or Defect

Change Definition Analyse Change Request and Faults Change Required

Impact Assessment No Change Required

Change Proposal

Documentation Impact

Alternative Proposal

Back-out Plan Authorization to Proceed

Change Review Board

Implement Change C a ge Proposal oposa Rejected

Change P Ch Proposall Accepted

Documentation Update

Communicate Decision

Initiate Change

Test Change Make Change Operational

Approve completed change

85

Delivering sustainable risk based solutions

Maintaining Documentation PROJECT G1

Idea, Feasibility & Business Case

Proposed Business Benefits, Business / IT Strategic Fit, Resources, Milestone Plan, Potential Project Risks

Project Stages

Regulatory + GPP

Initial Planning

Budget resource and technology costs Return on Investment, Initial Project Plans

Good Project Practice

Typical Vendor Activity

Initial Risk Assessment

G2

Portfolio Management

G3 Project Definition & Organisation

Portfolio Review

G5

Detailed Planning

Business/Manufacturing Processes

Criticality of Business / Manufacturing Processes

Product evaluation, Supplier demonstrations Technical fit, Requirements fit, Validation implications Vendor Proposals / Tenders, Business , Quality and Commercial Evaluation and Selection of Solution

Vendor Assessment

Quality Organisation, Systems and Application Technical Capability, Experience & Resource Availability, Support Capability

Project Plan

Work breakdown, Scope, Structure, Timescale plan, Critical path analysis, Resources

Contract

Terms and Conditions Phasing, Organisation, Fixed Price, Reimbursable

Validation Plan

Validation Roles, Approach, Controls, Deliverables DJO vs. Vendor Activities and Deliverables

Configuration, fields, screens, reports Customisations, new functionality, database schema Servers, Data Storage Operating System, Security/Backup

Detailed Software Spec Requirements Traceability Hardware Specification Build Solution

Code & Configure Development Testing

G8

Acceptance

G9

Go Live

Test Planning

Unit/Module Specifications Software structure and logic

Internal Audit

Process flow, Functionality, Reports, Data Input Processing, Communications, Exception Handling, Database schema, etc

Infrastructure Specification ERES Risk Assessment

Change Management

Functional Specification Configuration Specification

Functional Risk Assessment

G7

Design Create & Maintain Document Management

Design & Design Review

Cost Breakdown, Investment Phasing Stage Payments

Maintain

Project Planning and Monitoring

Design

Definition and Analysis of User, Functional and Technical Requirements

Solution Evaluation Request for Proposal

Risk and Deviation Management

G6

Requirements

Importance vs. other project commitments, Capacity to deliver, commitment to resources Stakeholder commitment, Project Go / No Go

Business Process Risk Assessment

Investment & Cost Plan

Configuration

System Construction, Interfaces Hardware Configuration, Cabling/wiring

Coding Standards, Code Reviews Structural Testing, Integration Testing Custom code Test Cases, Objectives, Phasing, Relationships, Test Failure Management, Technical and Human Resources

IQ, OQ, PQ Interim Reports

Installation verification, Challenge Testing Functional Testing, Performance / Process Testing

Data Migration/ Data Input

Data Cleansing, Migration Planning, Migration Testing

Training Production IQ

Handover

Handover to Operational and Support Organisations

Conformance to Validation Plan

Post Implementation Monitoring

Process and System Effectiveness and Stability

G11

Operate

Change Control, Configuration Management, Security, Backup and Restoration, Disaster Recovery, Business Continuity, Maintenance, Periodic Review

G12

Retire

Project Learning Review

Test Protocols

Hardware & Software Installation

Validation Report

G10 Project, System / Audit & Review

CHANGE CONTROL

Initial Assessment of Project Risks, Regulatory Risks and Project Commitment

Business Process Mapping

Requirements Management

G4 Solution Evaluation

Validation Plan

Initial Project Definition

Planned vs. Actual, Benefits, Plan, Resource, Contract Strategy Quality Strategy, Requirements

Software Archive Records Archive & Retention

Validation Report Delivering sustainable risk based solutions

43

9/13/2010

Emergency Changes Very occasionally it is necessary to implement changes quickly to minimise risk to patient safety or business operation.

1) Consult users to discuss and agree

2) Do change 3) CC completed usually within 24 hours

4) Consider “holds” based on impact e.g. product not released until CC reviewed and signed off Delivering sustainable risk based solutions

Incident Management f System failures failures, bugs bugs, anomalies anomalies, errors recorded on an Incident Report, System Logbooks or in an appropriate Error Log.

f Issues must be reviewed and appropriate action taken to rectify the problem.

f Incident Reports / Error Logs periodically reviewed to determine trends indicating inadequate system design, implementation or testing.

88 Delivering sustainable risk based solutions

44

9/13/2010

Security Management f Maintain integrity of data f Maintain availability of Information Assets f Maintain confidentiality and to restrict accessibility (physically and logically) of information and system functionality to authorized personnel only. f Support business continuity processes.

89 Delivering sustainable risk based solutions

User Access Controls

90 Delivering sustainable risk based solutions

45

9/13/2010

Password Reset f If a user forgets their password they must request a System Administrator reset. f Before implementing the reset, the System Administrator must positively identify the user. System Administrator knows the person requiring reset and is able to identify them (e.g. by voice) when a reset is requested User’s line manager / supervisor sends and email to System Administrator confirming identity of requester Person requesting reset presents themselves to System Administrator with picture ID

f System is implemented to allow use of characters from secret words f User must change their password the first time they log onto the system following reset (providing system has the capability).

91 Delivering sustainable risk based solutions

Periodic Review f Periodic Reviews of regulated g computerized p systems y shall be conducted in order to ensure systems are maintained in a validated state. Periodic Review shall examine: Maintenance of documentation and records in line with system changes Reported issues / incidents Evidence of unstable or unreliable operation Changes in environment, process or business requirements, regulation or accepted best practice Level and type of changes being made Outstanding actions from Validation Report, Audit Reports and Previous Periodic Reviews Change of system use Security controls Personnel (training of new users or persons changing roles)

f Periodic Review shall be conducted annually for critical systems (e.g. patient safety related) and every 3 years for non critical regulated systems (e.g. training systems).

f Periodic Review of regulated IT Infrastructure shall be conducted every 2 years.

92 Delivering sustainable risk based solutions

46

9/13/2010

Data Migration Planning f Approaches Re-keying of records into new system Custom data migration programs Inbuilt tools within application

f SOPs required for manual data migration operations f Validation of custom Data Migration Programs f Off-the-shelf data migration tools validation in accordance with risk f Data Cleansing Delivering sustainable risk based solutions

Purpose

Automated Routines

Validation

Sample to enable testing

Phase D Development l tT Testing ti

Increased sample

Use

Synchronization with legacy systems

Qualification

Go Live

Delivering sustainable risk based solutions

47

9/13/2010

Legacy System

New System SOP/Manual Verification

Validation Table A’

Table A

Manual Translation Routines

Automated Translation Routines

Table B’

Table B Table C’ Translation Scheme

Delivering sustainable risk based solutions

Data Migration Verification f Data migration verification tests shall be documented to describe the checking process. The sample of data verified shall be determined by: Scope of data to be migrated Criticality of data to be migrated Degree of data translation between systems Risk of data error arising from the data migration approach (e.g. upgrade to new version of same product, custom data migration tools, manual data entry)

96 Delivering sustainable risk based solutions

48

9/13/2010

Legacy Systems f Legacy computerized systems are those systems that are in use and may not adequately comply with this policy, regardless of age.

f Documented Risk Assessments shall be conducted to determine the criticality of any identified shortfalls.

f Remediation plans shall be established to address unacceptable risks. Remediation shall be focused on demonstrating: System specification to a level that supports understanding and acceptance of system processes and functions Suitable platform for system use, change, maintenance and disaster recovery (e.g. g specifications, p , system y specifications) p ) configuration System processes and functions operate and perform in accordance with approved specifications (e.g. testing) System performance, stability and free of known critical faults

97 Delivering sustainable risk based solutions

GAMP categories and spreadsheets f Category 1 - Product on which the spreadsheet is developed f Category 2 – N/A f Category 3 – Use of native spreadsheet functions with no configuration

f Category 4 – Initial input of data requires the application to automatically branch to different cells to perform the data manipulation

f Category 5 – Custom macros or nested logic/lookup functions 98 Delivering sustainable risk based solutions

49

9/13/2010

Disposable Spreadsheets f Intended to be used only once f Only for “simple” applications f Print the spreadsheet showing: Formula Input data Output results

f Calculations must be verified. Attach hardcopy verification evidence as to the printed spreadsheet 99 Delivering sustainable risk based solutions

Internal Audit f Internal functions audit other functions (or disciplines)

f Focus on a specific area of the QMS f Keep scope sensible f Observations more practical and value added

f Better ownership of observations

100 Delivering sustainable risk based solutions

50

9/13/2010

Common Pitfalls / Audit Observations – Docs / Plans / SOPs f Documentation Failure to follow templates Templates too rigid No justification for not addressing a template requirement Documents not approved Documents not under configuration management

f Plans and SOPs Don’t follow the plan or SOP Don’t raise deviations when plan or SOP not followed SOP assigns roles to people who don’t have authority to execute role Not prescriptive enough so difficult to follow Too prescriptive so difficult to follow!!!! Not practical 101 Delivering sustainable risk based solutions

102 Delivering sustainable risk based solutions

51

9/13/2010

Common Pitfalls / Audit Observations - Specifications f f f f f f f f f f f f

Requirements not clear clear, concise concise, measurable Defined too early or too late Project team ignore URS Focus on product rather than business processes Not maintained during the implementation phase Not tested Not created Developed by wrong people Design not traceable to Requirements Deviations from requirements not managed Design evolves after issue of Requirements or Design Documentation Design issues are documented but not addressed 103 Delivering sustainable risk based solutions

Common Pitfalls / Audit Observations - Testing

y Specifications

Acceptance criteria

y No test specifications y Lack of approvals (pre, test,

post)

y Methods y y y y

• Not N td defined fi d • Ambiguous • Embedded in method

Results

Not repeatable Lack of boundary challenges Not traceable to design g Inadequate deviation management

• Inappropriate Records • Do not clearly demonstrate that p criteria met acceptance

• Not supported (where required) by raw data

• Raw data not signed • Raw data not referenced to test

Delivering sustainable risk based solutions

52