《AFrameworkforNCCLSEvaluationProtocols-AReport(EP19-R).pdf》由会员分享，可在线阅读，更多相关《AFrameworkforNCCLSEvaluationProtocols-AReport(EP19-R).pdf（9页珍藏版）》请在得力文库 - 分享文档赚钱的网站上搜索。
1、Volume 22 EP19-RiA Framework for NCCLS Evaluation Protocols; A Report Abstract This report describes the different types of performance studies that are conducted to evaluate clinical assays: namely, validation, verification, and demonstration studies. Each evaluation protocol guideline is categoriz
2、ed with respect to its evaluation type. The tasks that comprise the path of workflow, which takes place when an assay is considered for use by a laboratory, are listed and explained. A recommendation for the data analysis portion of the evaluation is discussed, whereby the use of confidence interval
3、s over hypothesis testing is recommended. NCCLS. A Framework for NCCLS Evaluation Protocols; A Report. NCCLS document EP19-R (ISBN 1-56238-464-3). NCCLS, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2002. THE NCCLS consensus process, which is the mechanism for moving a docum
4、ent through two or morelevels of review by the healthcare community, is an ongoing process. Users should expect revisededitions of any given document. Because rapid changes in technology may affect the procedures,methods, and protocols in a standard or guideline, users should replace outdated editio
5、ns with thecurrent editions of NCCLS documents. Current editions are listed in the NCCLS Catalog , which isdistributed to member organizations, and to nonmembers on request. If your organization is not amember and would like to become one, and to request a copy of the NCCLS Catalog , contact theNCCL
6、S Executive Offices. Telephone: 610.688.0100; Fax: 610.688.0700; E-Mail: exofficenccls.org;Website: www.nccls.org Volume 22 EP19-RxiiiContents Abstract. iCommittee Membership. vActive Membership. viiForeword. xv1Introduction. 11.1Purpose.11.2Consensus on Statistical Application. 11.3Literature Citat
7、ions. 12Scope. 23Definitions. 24Path of Workflow. 35Phase 1: Plan.35.1Define Requirements. 35.2Select Test System to Be Evaluated.35.3Scope of the Evaluation Study. 35.4Selection of the Study Protocol. 66Phase 2: Perform Testing or “Do”.66.1Confidence Intervals Versus Hypothesis Testing. 66.2Hypothe
8、sis Tests. 86.3Confidence Intervals. 87Phase 3: Check (and Communicate Conclusion). 108Phase 4: Implement or Act. 119Conclusion. 11References.12Related NCCLS Publications. 13Volume 22 EP19-RxvForeword This report presents an overview of the relationships among evaluation experiments that are perform
9、ed by manufacturers, by clinical laboratories, and/or that may be required by governmental agencies. It is the objective of the NCCLS Area Committee on Evaluation Protocols to provide recommendations for the assessment of laboratory test procedures in a way that defines good laboratory practices tha
10、t are grounded on good scientific and statistical theory. NCCLS evaluation protocols are primarily intended to be practical in their use in the clinical laboratory. All clinical laboratories must perform method evaluation studies prior to using a new or revised test system for reporting patient test
11、 results. The extent of a given study may vary considerably depending on the objectives of the laboratory, on resources available in the laboratory, and on information that may have been previously reported in technical literature by other laboratories. NCCLS evaluation protocols are not intended to
12、 be so comprehensive as to provide the sole data on product testing and qualification on the part of manufacturers of in vitro diagnostic devices. There are many other studies that a manufacturer will perform during the research and development phases and during the manufacturing validation phase th
13、at are unique to the design of the test kit or test system and the manufacturing processes themselves. These special studies go beyond the scope of NCCLS evaluation protocols. However, it is appropriate that manufacturers use these NCCLS documents as a means of generating performance data for the fi
14、nal product in a form to which the laboratory can relate. In addition to defining a framework for a family of evaluation studies that should result in greater consensus and standardization of practice, this report reviews two different approaches for statistical analysis of the results: tests of sig
15、nificance and the use of confidence interval estimates. The latter is preferred, so that the laboratorian focuses attention on the magnitude of an error assessment in the comparison to stated claims or stated requirements from a clinical perspective. EP19 was originally conceived as a report and was
16、 primarily intended for use by evaluation protocols subcommittees. However, as draft copies were circulated during its preparation, some NCCLS users outside of the evaluation protocols area committee who reviewed the report found intrinsically valuable information in the document and suggested that
17、it be made available as a guideline. The Area Committee on Evaluation Protocols concurred with users recommendations and issued EP19 as a proposed-level consensus document. However, due to the limited input received on EP19-P during the public review and comment period, the area committee agreed tha
18、t EP19 be reissued as a report, made available to all NCCLS users. Key Words Demonstration, evaluation protocol, validation, verification Number 10 NCCLSxviThe Quality System Approach NCCLS subscribes to a quality system approach in the development of standards and guidelines, which facilitates proj
19、ect management; defines a document structure via a template; and provides a process to identify needed documents through a gap analysis. The approach is based on the model presented in the most current edition of NCCLS document HS1A Quality System Model for Health Care. The quality system approach a
20、pplies a core set of “quality system essentials (QSEs),” basic to any organization, to all operations in any healthcare services path of workflow. The QSEs provide the framework for delivery of any type of product or service, serving as a managers guide. The quality system essentials (QSEs) are: EP1
21、9-R Addresses the Following Quality System Essentials (QSEs) Documents&RecordsOrganizationPersonnelEquipmentPurchasing&InventoryProcessControlInformationManagementOccurrenceManagementAssessmentProcessImprovementService&SatisfactionFacilities&SafetyX Adapted from NCCLS document HS1A Quality System Mo
22、del for Health Care.QSEs Documents & RecordsInformation ManagementOrganization Occurrence Management Personnel Assessment Equipment Process Improvement Purchasing & Inventory Service & Satisfaction Process Control Facilities & Safety Volume 22 EP19-RAn NCCLS global report. ?NCCLS. All rights reserve
23、d.1A Framework for NCCLS Evaluation Protocols; A Report 1 Introduction It is the objective of the NCCLS Area Committee on Evaluation Protocols to provide recommendations for the assessment of laboratory test procedures in a way that defines good laboratory practices that are grounded on good science
24、 and statistical theory. These recommendations are primarily intended to be practical in their use in the clinical laboratory. They are not intended to be so comprehensive as to be the sole data on product testing and qualification on the part of manufacturers of in vitro diagnostic devices. However
25、, it is appropriate that manufacturers use these documents as a means of generating performance data to which the laboratory can relate. 1.1 Purpose It is recognized that laboratories may have different needs and purposes for conducting evaluation studies; they may have different resources to apply
26、to an evaluation study; and there may be different performance requirements. This report creates a framework that relates the scope and design of an evaluation study to the purpose of the study. The framework also integrates this hierarchy of evaluation studies with the amount of performance informa
27、tion that may be known prior to the study. (This is discussed in greater detail in Section 2, Scope.) Finally, this report incorporates the fundamentals of a defined path of workflow1- 3 for an evaluation study as described in Section 4. Please refer to the most recent edition of NCCLS document HS1A
28、 Quality System Model for Health Care for additional information. All evaluation studies, regardless of purpose and scope have certain key elements or steps in common. Key issues that have been sources of confusion are discussed in more detail in the appropriate steps in the overall process for an e
29、valuation study. 1.2 Consensus on Statistical Application Implied in the discussion of purpose is the requirement that there be a common understanding of the best approach in the application of statistics for data analysis and interpretation. Section 6.1 states that the use of confidence interval es
30、timates of error is preferred over tests of statistical significance.4-10 Confidence interval estimates reflect both the magnitude of an error as well as the uncertainty of that estimate, while tests of significance are based on a ratio of errors, so that the magnitude of the error may be lost in th
31、e final interpretation. It is recognized that in the clinical laboratory setting, the magnitude of the error due to a given effect is of greater importance than whether the error is “statistically” significant. The ultimate decision regarding the acceptability of the performance of a given test or p
32、roduct should be based on whether the magnitude of a given error is less than the medical or regulatory requirement. 1.3 Literature Citations It is recommended that statistical methods, whether for experimental design, data analysis, or interpretation of data, be referenced from peer-reviewed journa
33、ls and/or established reference textbooks. Several NCCLS guidelines have been published with little or no reference to the technical literature, and this should be corrected. New documents and those under development should be adequately referenced prior to the next stage in the consensus process. E
34、xisting documents should have appropriate references added at the time of their next review. Number 10 NCCLSAn NCCLS global report. ?NCCLS. All rights reserved.22 Scope This report is intended for use by all constituent groups of NCCLSclinical laboratory professionals, manufacturers, and governmenta
35、l agenciesas a basis for arriving at a consensus for the need for different evaluation studies that are designed to meet different purposes. NCCLS evaluation protocols are not intended to be so comprehensive as to provide the sole data on product testing and qualification on the part of manufacturer
36、s of in vitro diagnostic devices. There are many other studies that a manufacturer will perform during the research and development phases and during the manufacturing validation phase that are unique to the design of the test kit or test system and the manufacturing processes themselves. These spec
37、ial studies go beyond the scope of NCCLS evaluation protocols. However, it is appropriate that manufacturers use these recommendations as a means of generating performance data for the final product in a form to which the laboratory can relate. 3 DefinitionsaAlpha level , n The probability that one
38、will reject the null hypothesis when it is true; NOTE: The alpha level is often set at 5 or 1 percent. Confidence interval, n An interval estimate of a population parameter computed so that the statement “the population parameter lies in this interval” will be true . in a stated proportion of the ti
39、mes such statements are made. Demonstration (study), n A study performed by a laboratory to show that it is capable of using a test system to obtain expected performance. Evaluation (study), n A generic term for any study that measures the performance capabilities of an assay.Hypothesis testing, n I
40、n Statistics, the testing of two or more statistical hypotheses that are mutually exclusive so that exactly one hypothesis can be accepted at a specified confidence level. Null hypothesis, n The hypothesis of no difference or no differential effects beyond random variation.Point estimate , n A value
41、 that summarizes a set of data without accounting for the precision of the estimate (e.g., its uncertainty). Validation , n 1) Confirmation, through the provision of objective evidence, that requirements for a specific intended use or application have been fulfilled; NOTES: a) The term validated is
42、used to designate the corresponding status (ISO 9000:2000); b) The use conditions for validation can be real or simulated; 2) A term used by the FDA for a study used to determine whether a test system meets user needs. (Federal Register: Part VII Dept. of Health and Human Services, FDA. 12 CFR Parts
43、 808, 812, and 820. Rules and Regulations. October 7, 1996;61(195):5222252601); 3) The action or process of proving that a procedure, process, system, equipment, or method used works as expected and achieves the intended result. Verification , n 1) Confirmation, through the provision of objective ev
44、idence, that specified requirements have been fulfilled; NOTES: a) The term “verified” is used to designate the corresponding status; b) Confirmation can comprise activities such as: performing alternative calculations; comparing a a Some of these definitions are found in NCCLS document NRSCL8Termin
45、ologyand Definitions for Use in NCCLS Documents. For complete definitions and detailed source information, please refer to the most current edition of that document. Volume 22 EP19-RAn NCCLS global report. ?NCCLS. All rights reserved.13Related NCCLS Publications*EP5-AEvaluation of Precision Performa
46、nce of Clinical Chemistry Devices; Approved Guideline (1999) . This document provides guidance for designing an experiment to evaluate the precision performance of clinical chemistry devices; recommendations on comparing the resulting precision estimates with manufacturers precision performance clai
47、ms and determining when such comparisons are valid; as well as manufacturers guidelines for establishing claims. EP6-P2Evaluation of the Linearity of Quantitative Analytical Methods; Proposed GuidelineSecond Edition (2001). This document provides guidelines for characterizing the linearity of a meth
48、od during a method evaluation; for checking linearity as part of routine quality assurance; and for determining and stating a manufacturers claim for linear range. EP7-P Interference Testing in Clinical Chemistry; Proposed Guideline (1986). This document provides background information and procedure
49、s for characterizing the effects of interfering substances on test results. EP9-AMethod Comparison and Bias Estimation Using Patient Samples; Approved Guideline (1995) . This document addresses procedures for determining the bias between two clinical methods or devices and for the design of a method
50、 comparison experiment using split patient samples and data analysis. EP10-APreliminary Evaluation of Quantitative Clinical Laboratory Methods; Approved Guideline (1998) . This document addresses experimental design and data analysis for preliminary evaluation of the performance of an analytical met
黑公网安备:91230400333293403D