Helen Cantwell is a chemist working in the Veterinary Toxicology Section of The State Laboratory. Her areas of interest are method development, validation, mass spectrometry and quality management. She is a member of the Eurachem Ireland secretariat and one of the Irish representatives on the Eurachem Method Validation Working Group. In the report below, she gives an overview of her visit to a Eurachem workshop 'Validation in Analytical Science – Current Practices and Future Challenges' held in Ghent, Belgium earlier this year. Funding for this visit was provided under the safefood Training & Mobility Programme. More information on the programme is available here.
Navigating the Challenges of Method Validation – with a little help from Eurachem
As an analytical scientist working in a regulatory laboratory, method validation is a part of my daily working life. In my field of veterinary drug residue analysis, all methods used must be accredited to ISO17025. There is also constant change. New drugs are licensed, permitted limits change or are newly introduced, new matrices must be analysed and new analytical technology is developed. Also, method validation is not a one-off thing. We cannot demonstrate that a newly developed method is fit for purpose, introduce it into routine use and then sit back happy in the knowledge that our work is done. A significant part of method validation involves demonstrating that our method remains fit for purpose while in routine use. Method validation is therefore an ongoing process which requires significant time and resources. It is important then that we, as analytical scientists, ensure that we have the knowledge and expertise to effectively carry out validation studies and to keep abreast of new techniques which may maximise the information we gather from validation experiments while minimising the experiments themselves. It was with this in mind that I became interested in a Eurachem workshop run in May of 2016 entitled Validation in Analytical Science – Current Practices and Future Challenges.
Eurachem is a network of organisations in Europe with members from 32 countries. It aims to establish a system of international traceability for chemical measurements and promotes good quality practices. Technical activity is carried out by a number of working groups which publish the highly respected Eurachem guides which are freely available on the website www.eurachem.org. The working groups also organise international workshops. In May of 2016, the Method Validation Working Group ran a workshop which took place over two days in Ghent in Belgium. The workshop consisted of a programme of lectures from international speakers, poster presentations and working group sessions where the workshop participants could discuss features of method validation, challenges or issues faced and ways in which these challenges may be overcome. The safefood Training and Mobility Programme kindly agreed to sponsor my attendance at this workshop so, in May of 2016, I and two of my colleagues headed off to Ghent.
The first thing I have to mention is that the facilities were wonderful. The conference room itself and the catering were all top class. The content also was highly applicable. It started with an overview of international guidance in the area of method validation. This presentation, which outlined the development of method validation requirements over the years, was given by Lorens Sibbesen, chair of the Method Validation Working Group. I was surprised to find that the concept of method validation in analytical chemistry, which is now so endemic, is a relatively recent one. Laboratories seeking accreditation before the advent of ISO 17025 worked to EN 45001 General criteria for the operation of testing laboratories (1989). This standard stated that, where it was necessary to employ test methods and procedures which are not-standard, that these shall be fully documented. Documented only. It wasn’t until the 90s that the term validation started to emerge, initially in a set of harmonised guidelines published by the pharmaceutical industry, the ICH guidelines from the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. This also introduced the idea that there were characteristics of an analytical method that a validation procedure should monitor.
Since then the concept of method validation in analytical chemistry has taken a firm hold - and rightly so. ISO 17025 the international standard documenting General requirements for the competence of testing and calibration laboratories incorporates a section on validation of methods and requires that the validation demonstrates that analytical methods are fit for purpose. Legislation and guidelines pertaining to method validation in different fields of analysis – food safety, pesticides, pharmaceuticals, water, environment, forensics etc., have been published. While very important for ensuring the quality of analytical results reported, lack of harmonisation in these guidelines and legislation has yielded further challenges which became obvious during some of the working group discussions. The Eurachem Guide The Fitness for Purpose of Analytical Methods (available free to download from the website www.Eurachem.org) attempts to combat this by documenting a generic approach to method validation with some references to sector specific guidelines. Lorens finished his presentation by highlighting the ongoing work of the Eurachem Method Validation Working Group. The group will be addressing challenges remaining in the field of method validation. These include validation of multi-parameter and multi-matrix methods, verification of test kits, instrument qualification and setting performance requirements.
Bertil Magnusson from the SP Technical Research Institute of Sweden addressed the topic of setting requirements for a method. Method validation involves demonstrating that our method is fit for purpose. We can only do this if we are clear on what our method is required to do. Our first task, therefore, is to discuss with our client and determine their needs. We then need to study relevant legislation and guidelines. When this is complete we can document our method requirements. These will include: 1) Measurement range – can our instrumentation cover the entire measurement range? Is there adequate sensitivity and linearity? May samples have to be diluted to bring them within the working range of the instrument? 2) Precision – how will precision be determined? Is a single laboratory validation sufficient or will an inter-laboratory study have to be carried out, for example if a method is being developed to be used as a standard method? 3) Trueness – is there a maximum bias that is acceptable? 4) Measurement uncertainty – is there a maximum uncertainty that is acceptable? When the requirements of the method have been determined and have been clearly documented, then we can assess whether our analytical method is fit for purpose.
Stephen Ellison of LGC in the UK continued with the theme of planning validation studies and addressed some frequently asked questions: How many performance characteristics should be studied? How many samples and replicates are needed? How can the necessary information be gathered with the minimum amount of work? Unfortunately, there is no single answer to these questions. The question of which performance characteristics to study revolves again around fitness for purpose. Validation of a method which aims to identify an analyte only may require the determination of selectivity/specificity whereas validation of quantitative methods must always incorporate a determination of trueness, precision and linearity. An assessment of detection capability is usually required while ruggedness testing, although usually carried out, is often incorporated into the method development process with only some fields insisting it must be addressed as part of the method validation procedure. The number of experiments, or observations, necessary to determine these characteristics differs in different sectors and the best way to determine this is to study the relevant legislation and guidelines. A statistician’s approach - that of using test power, is being documented in the IUPAC guidance document, Experiments for Single Laboratory Validation of Methods of Analysis: Harmonized Guidelines, which is currently in draft. The test power is defined as the probability of rejecting the null hypothesis when it is false. The approximate precision, the size of bias required and the confidence level must all be known or assumed. The number of observations necessary can then be calculated and these observations carried out in whatever combination of experiments and levels is deemed appropriate. However, because of the number of assumptions which must be made, use of test power must be approached cautiously. It is useful for comparing different validation designs.
Efficient experimental design is the way to maximise the information gathered from the minimum amount of work. Firstly, it is possible to determine more than one performance characteristic from a single set of experiments. Legislation published in the area of veterinary drug residue analysis, 2002/657/EC describes a set of three experiments with 7 observations at each of three concentration levels in each experiment. From this single set of experiments precision at three concentration levels, bias at three concentration levels, an assessment of linearity and determination of detection capability can be calculated. Similarly, a factorial design can be used to minimise the number of experiments required to determine the ruggedness of a method.
Sarah de Saeger of the University of Ghent runs an accredited laboratory in an academic environment. Accreditation was initially sought as Sarah’s laboratory also carries out drug residue and contaminant testing in food and animal feed for which accreditation is required. The extent of method validation, including proficiency testing, required of an accredited lab may often cause academics to balk due to the high cost in both time and money. What benefit then, if any, does implementing a quality system with stringent validation protocols have in a university laboratory? A primary function of a university is to teach. Carrying out research in such a laboratory will ensure the production of highly trained graduates who are perfectly placed to integrate into the workplace. Another function of a university is to carry out research. The presence of a quality system means that the research will generate reliable results and high quality data. The benefits seen in Sarah’s laboratory include an increase in the number of projects funded and publication in journals with high impact factors.
Method validation in accreditation was discussed in a very interesting talk given by a representative of BELAC, the Belgian equivalent of our own Irish National Accreditation Board. ISO 17025 General requirements for the competence of testing and calibration laboratories is currently being updated. It is due to come into effect in 2017 and some significant changes are proposed. The concept of verification is being introduced; this will align it with ISO 15189 Medical laboratories – requirements for quality and competence. Requirements for sampling will be increased and the role of the laboratory as a conformity assessment body will be addressed. We therefore need to be prepared for changes, and, quite possibly the need to update our method validation procedures.
Along with a poster session and other presentations covering validation in clinical chemistry and of microbiological methods, inter-laboratory validation and the use of both Quality by Design and chemometrics, the workshop included two working group sessions where participants discussed topics relating to method validation. I attended a working group session on the determination of trueness/bias and one on challenges in validation of multi-parameter methods. From participation in these working groups it became clear to me that one of the main challenges in method validation is the lack of harmonisation across sectors. The attendees at the working group sessions were from many areas of analytical science. When determining trueness/bias we follow different guidelines, are bound by different legislation, use differing numbers of observations and have different criteria for acceptance and rejection. In the working group session on validation of multi-parameter methods it was obvious that even basic terminology was different in different fields. The official language of the conference was English and I am a native speaker and yet I found myself having to translate before speaking. I was translating from those terms I, working in veterinary drug residue analysis, use everyday such as maximum residue limit and CCalpha values to those the other participants would understand – permitted limit and level at which a sample is considered non-compliant. We had to clarify our terminology before we could discuss anything relating to multi-parameter methods. Once we did we agreed that the challenges faced included a lack of certified reference materials, a lack of proficiency testing, validation in multiple matrices and the cost and time involved.
Method validation is an essential part of analytical chemistry and will continue to constitute a large part of the workload of the analytical chemist. Development of new technologies, movement towards multi-analyte and multi-matrix methods and changes in legislative and accreditation requirements mean that procedures for method validation must constantly evolve. The Eurachem Method Validation Working Group is an important component of this evolution, functioning as a centre of expertise and assembling and promoting best practice. I have decided to become involved and have recently joined the working group as one of the Irish representatives. I would be happy to raise any concerns or queries Irish scientists may have at the working group meetings so please feel free to contact me.
All of the presentations, posters and reports from the working group sessions are available on the Eurachem website www.eurachem.org.
I gratefully acknowledge the financial sponsorship of safefood in attending this workshop.
Comments