Validation of Alternative Methods Food Labs in Crystal Ball – Future Challenges in Food Analysis, 21-22 May 2015 AOAC EUROPE - NMKL - NORDVAL INTERNATIONAL SYMPOSIUM 2015

Dr Gail Betts Campden BRI, Chipping Campden, Gloucestershire, GL55 6LD, UK [email protected]

Why am I interested in Method Validation? • I work in the Microbiology Department at Campden BRI in UK

• We hold ISO 17025 accreditation for a range of microbiological tests and test for a large number of clients • We have 2400 member companies in 75 countries who rely on us for advice. Particularly on testing. • We help companies develop and implement new test methods • We are an expert laboratory for validation bodies such as NordVal, MicroVal & AOAC RI (30 validations)

Validation of Alternative Methods • There are currently many different test methods available, – for example in a review of rapid methods we found over 70 methods available for Salmonella!

• So how does a laboratory decide which method to use…..In many cases the validation status is the most crucial factor of choice

• This talk will examine – – – –

What is method validation Why it is important to validate How is validation done (according to ISO 16140) Things to look out for when doing a validation

What is method validation? ‘validation’ establishment of the performance characteristics of a method and provision of objective evidence that the performance requirements for a specified intended use are fulfilled EN ISO 16140-1 Microbiology of food and animal feed - Method validation Part 1: Vocabulary

How is method Validation done? ISO 16140 – part 2

Why is method validation important? • European Food Law : Food Business Operators (FBOs) are obliged to comply with the microbiological criteria set out in Commission Regulation 2073/2005 CRITERIA •Food category •Microorganism •Sampling plan •Limits •Method to be used •Stage of production where criterion applies •Action to be taken if results unsatisfactory www.fsai.ie/uploadedFiles/Consol_Reg2073_2005.pdf

2073/2005- Alternative methods equivalency Article 5—2073/2005 • The use of alternative methods is acceptable when the methods are validated against the reference method in Annex 1 and if a proprietary method, certified by a third party in accordance with the protocol set out in EN/ISO 16140 or other internationally accepted similar protocols • If the FBO wishes to use analytical methods other than those validated and certified as described in paragraph 3 the methods shall be validated according to internationally accepted protocols and their use authorised by the competent authority

Third Party Validation Systems No single validation system that satisfies users throughout the world – although there are many ‘harmonisation talks on-going Europe NordVal AFNOR MicroVal USA

AOAC International Official Methods Program AOAC Research Institute Performance Tested Methods Program

•

ISO Standard (EN ISO 16140): Microbiology of food and animal feeding stuffs- Protocol for the validation of alternative methods

EN ISO 16140

SCOPE •This part of ISO 16140 specifies the general principle and the technical protocol for the validation of alternative, mostly proprietary, methods in the field of microbiological analysis of food, animal feed, and environmental and primary production stage samples.’ •‘Validation studies according to this standard are intended to be performed by organisations involved in method validation

ISO 16140:2013 : Quantitative Methods Method Comparison • 5 food categories for a ‘broad range of foods’ Relative trueness • 5 samples of 3 food types per 5 categories = 75 samples Accuracy profile • 3 contamination levels per category with 2 samples at each level with 5 replicates = 150 samples Inclusivity/Exclusivity (not applicable to total count determinations) • 50 target strains - 30 non target strains Inter-laboratory Study • Min - 8 sets of lab results • 1 food, 3 levels, 2 replicates (min)

Food Categories (ISO 16140) 15 food categories e.g. • Meat –raw, RTE, RTRH • Poultry – raw, RTE, RTRH • Fish - raw, RTE, RTRH • Produce, fresh, processed, dried • Dairy – raw and heat processed • Chocolate/bakery/confectionary • Multi-components foods • Pet food/Animal feeds • Environmental and Primary production

ISO 16140‘new’ version : Quantitative Methods Method Comparison : Relative trueness • 5 samples of 3 food types per 5 categories = 75 samples Category Dairy products

Fruits and vegetables

Raw poultry and meats

Ready to eat foods

Multi component foods

Types Dairy desserts e.g. chilled custard, trifle Soft cheese Hard cheese e.g. cheddar Seasonings e.g. spices Sprouts e.g. mung beans Leafy greens e.g. parsley, lettuce Fresh chicken cuts Fresh mince Frozen patties Ready to eat poultry e.g. turkey fillet, chicken sausage Cooked fish products e.g. prawns Cooked meat e.g. ham Composite foods with raw ingredients e.g. sandwiches, pasta salads. Mayonnaise based salads Cooked chilled foods e.g. rice products

Number of samples 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

ISO 16140 ‘new’ version : Quantitative Methods Method Comparison : Accuracy profile • 3 contamination levels per category with 2 samples at each level with 5 replicates = 150 samples Types

Dairy products

Dairy desserts

Item

Level

Chilled custard Low:100cf/g Medium : 1000cfu/g High : 10,000cfu/g

Fruits and vegetables

Raw poultry and meats

Leafy greens e.g. parsley, lettuce

Parsley

Fresh beef

Fresh steak

Low: 50cf/g Medium : 1000cfu/g High : 50,000cfu/g Low: 50cf/g Medium : 1000cfu/g High : 50,000cfu/g

Ready to eat foods

Cooked fish products e.g. prawns

Fish pate

Low: 50cf/g Medium : 100cfu/g High : 1000cfu/g

Multi Composite component foods with foods raw ingredients

Pasta salad

Low 500cf/g Medium : 5000cfu/g High : 50,000cfu/g

Test portion 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2 1-5 Batch1 1-5 Batch2

Fresh produce 0,60 Bias

0,40

β-ETI

0,20 Bias

Category

0,00 0,00 -0,20

AL = +/- 0.5 2,00

4,00

-0,40 -0,60

Reference Median

6,00

ISO 16140 : Qualitative Methods overview Stage 1: Comparison study • 5 food categories • 60 samples per category (~ 50% positive) Inclusivity/Exclusivity • 50 target organisms (100 for Salmonella) • 30 non target organisms • specificity, sensitivity and relative accuracy Relative Level of Detection Stage 2: Inter-laboratory Study • 10 sets of lab results • 1 food, 3 levels, 8 replicates Responses

Reference Method Positive (R+)

Reference Method Negative (R-)

Alternative method positive (A+)

+/+ positive agreement (PA)

-/+ positive deviation (PD) (R-/A+)

Alternative method negative (A-)

+/- negative deviation (ND) (A-/R+)

-/- negative agreement (NA)

NordVal

AOAC RI

MicroVal

AFNOR

Technical rules

√

√

√

√

Based on ISO 16140

√

√

√

√

Broad range

Broad range

√

√

Food Claims ILS

Broad range

√

Specific foods

×

Protocol written by

Expert lab

Technical consultants

Expert lab

Expert lab

Acceptance criteria

In NordVal protocol

Not defined

ISO 16140:2013

Not defined

Certificates renewed

2 years

1 year

4 years

4 years

Be aware that • Validation of water methods is not covered in ISO 16140 • Separate standard covers waters – ISO 17994

• Different experimental design and different statistics.

Compact Dry ETC for water Campden BRI & NordVal 2014 • Combined experimental design and statistics of ISO 16140 and ISO 17994 • (i) Campden BRI as the expert lab did typical 5 products x 5 replicates x 5 contamination levels • (ii) 8 collaborative labs tested 3 levels x 2 replicates • ISO 16140 statistics was done on linearity and accuracy data from (i) • ISO 17994 statistics was done on whole data set from all the labs

Compact Dry ETC for water

Validation of confirmation methods • Is currently not covered in ISO16140 but part 6 will cover this • ISO16140-6 'Microbiology of the food chain - Method validation - Part 6: Protocol for the validation of alternative methods for microbiological confirmation and typing' • AOAC : Appendix J – does include Confirmatory Identification methods

Things to look out for – expert laboratory • Even though all accreditation schemes are based on ISO 16140- each one has its own technical protocol so make sure you get the most up to date copy before you start writing a protocol • Manage the expectations of the kit manufacturer if they are not familiar with the validation time-frame. It takes on average 12 months for a validation to be completed • If doing a joint validation, start discussions with both schemes as early as possible to design a single acceptable protocol

Things to look out for : kit manufacturer • Make sure the kit is finalised before the validation starts : do not use a validation as a method development exercise • Do not change test method or kit instructions during a validation. This could cause delays or extra costs or could invalidate the study • Be familiar with the scheme rules as these will highlight the manufacturers responsibility • If existing validation was done under ISO16140 :2004, then you may need to do further studies when it is due for renewal to meet ISO16140 :2013

Things to look out for : end user • If testing under legislation and wanting to use a proprietary method – Don’t just look at a Third party website and use any ‘validated’ for the organism of interest.

• Check the validation certificate and report and ensure that the scope of the validation covers both the food types and organisms you wish to test for. – Issues- examples • There is at least one method in a certification bodies list of validated methods that is noted not to detect non-motile Salmonella– but is validated and certified to detect motile Salmonella. • Always check the food groups you wish to test have been included as a part of the validation – ‘all foods’ or specific matrix claim • Always verify a validated methods use within your laboratory

Final thoughts........Importance of Validation to an end-user “An official validation based on ISO 16140…is a must for our laboratories…. the food categories are very important for us…therefore we work with suppliers to validate our matrices…As for inclusivity test, because of some historical data, we are interested to have some specific strains on the list”

Further information on third party schemes:

AFNOR: www.afnor-validation.com

MicroVal: www.microval.org

NordVal : www.nmkl.org

AOAC: www.aoac.org

Please get in touch.... [email protected]