ASTM D6122-23 pdf download – Standard Practice For Validation Of The Performance Of Multivariate Online, At-Line, Field And Laboratory Infrared Spectrophotometer, And Raman Spectrometer Based Analyzer Systems.
The operation of a laboratory or process stream analyzer system generally involves five sequential activities:
- Correlation – Before starting the procedures outlined in this practice, a multivariate model is developed that connects the spectrum produced by the analyzer to the Primary Test Method Result (PTMR).
- (1a) If the analyzer and the Primary Test Method (PTM) analyze the same material, the multivariate model directly correlates the spectra to the PTMRs collected from those samples.
- (1b) If the analyzer measures the spectra of a material that has undergone treatment prior to PTM analysis, the multivariate model relates the spectra of the untreated sample to the PTMR for that same sample post-treatment.
- Analyzer Qualification – When an analyzer is first installed or after significant maintenance, diagnostic testing is carried out to confirm that the analyzer meets the manufacturer’s specifications and historical performance standards. These diagnostic tests may necessitate adjustments to the analyzer to ensure it produces predetermined output levels for specific reference materials.
- Local Validation – A local validation is conducted using an independent, albeit limited, set of materials that were not included in the correlation activity. This validation aims to demonstrate that the agreement between the Predicted Primary Method Test Results (PPTMRs) and the PTMRs aligns with expectations based on the multivariate model.
- General Validation – Once a sufficient number of PPTMRs and PTMRs have been collected from materials not involved in the correlation activity, and which adequately cover the compositional space of the multivariate model, a comprehensive statistical assessment can be performed. This assessment aims to show that the PPTMRs agree with the PTMRs within user-defined criteria.
- Continual Validation – Following a successful local or general validation, quality assurance control chart monitoring is implemented to track the differences between PPTMRs and PTMRs during the normal operation of the process analyzer system. This monitoring ensures that the agreement established during the General Validation is maintained.
This practice specifically addresses the third, fourth, and fifth activities in this sequence.
1.1 This practice outlines the requirements for validating measurements obtained from laboratory, field, or process (online or at-line) infrared (near- or mid-infrared) and Raman analyzers. These measurements are used to calculate physical, chemical, or quality parameters (i.e., properties) of liquid petroleum products and fuels, based on spectroscopic data and multivariate modeling methods. The validation process includes confirming adequate instrument performance, ensuring the calibration model is applicable to the sample spectrum, and verifying that the uncertainties associated with the results from infrared or Raman measurements align with user-specified requirements. Initially, a limited number of validation samples that represent current production are utilized for local validation. Once there are enough validation samples with sufficient variation in both property levels and sample compositions to cover the model calibration space, the statistical methodology outlined in Practice D6708 can be employed for general validation across the analyzer’s entire operating range. If adequate variation in properties and composition is not achieved, local validation will continue to be utilized.
1.1.1 For certain applications, both the analyzer and the PTM are applied to the same material. In this case, the multivariate model applied to the analyzer output (spectrum) directly generates a PPTMR for the same material from which the spectrum was obtained. The PPTMRs are then compared to the PTMRs measured on the same materials to assess agreement.
1.1.2 In other applications, the material analyzed by the analyzer system undergoes a consistent additive treatment before being evaluated by the PTM. The multivariate model applied to the analyzer output (spectrum) generates a PPTMR for the treated material, which is compared to the PTMRs measured on the treated materials to assess agreement.
1.1.3 In some instances, a two-step procedure is followed. The first step involves applying both the analyzer and PTM to measure a blendstock material. In the second step, the PPTMRs obtained in Step 1 serve as inputs to a second model that predicts the results of applying the PTM to the finished blended product produced by adding additives to the blendstock. If the analyzer used in Step 1 is a multivariate spectroscopic analyzer, this practice is employed to evaluate the agreement between PPTMRs and PTMRs. If not, Practice D3764 is used to compare the PPTMRs to the PTMRs for the blendstock. Since the second step does not utilize spectroscopic data, its validation is conducted using Practice D3764. If the first step employs a multivariate spectrophotometric analyzer, only samples whose spectra are not outliers in relation to the multivariate model will be used in the second step. It is important to note that the second model may account for varying levels of additive material added to the blendstock.
1.2 Typically, multiple physical, chemical, or quality properties of the sample being tested are predicted from a single spectral measurement. In applying this practice, each property prediction is validated independently. While the separate validation procedures for each property may share common features and be influenced by similar effects, the performance of each property prediction is assessed on its own. Users generally run multiple validation procedures concurrently.
1.3 The results used for analyzer validation come from samples that were not part of the multivariate model development and from spectra that are neither outliers nor nearest neighbor inliers relative to the multivariate model.
1.4 If the number, composition range, or property range of available validation samples do not adequately cover the model calibration range, local validation will be performed using samples that are representative of current production. When the number, composition range, and property range of available validation samples become comparable to those of the model calibration set, a general validation can be conducted.
1.4.1 Local Validation:
1.4.1.1 The calibration samples used in developing the multivariate model must exhibit sufficient compositional and property variation to establish a meaningful correlation and must cover the compositional range of samples to be analyzed using the model, ensuring that analyses are conducted via interpolation rather than extrapolation. The Standard Error of Calibration (SEC) indicates how well the PTMRs and PPTMRs align for this calibration sample set. SEC accounts for contributions from spectrum measurement error, PTM measurement error, and model error, with sample (type) specific biases being part of the model error. Typically, spectroscopic analyzers are highly precise, resulting in small spectral measurement error compared to other error types.
1.4.1.2 During initial analyzer validation, the compositional range of available samples may be limited compared to the calibration set. Due to the high precision of spectroscopic measurements, the average difference between PTMRs and PPTMRs may reveal a statistically observable sample (type) specific bias that is still less than the uncertainty of PPTMR, U(PPTMR). Consequently, the bias and precision of the differences between PTMRs and PPTMRs are not utilized as a basis for local validation.
1.4.1.3 Based on SEC and the leverage statistic, the uncertainty of each PPTMR, U(PPTMR), is calculated. During validation, for each non-outlier sample, a determination is made as to whether the absolute difference between PPTMR and PTMR, | |, is less than or equal to U(PPTMR). Records are kept of the total number of non-outlier validation samples and the number of samples for which | | is less than or equal to U(PPTMR). Given the total number of non-outlier validation samples, an inverse binomial distribution is utilized to calculate the minimum number of results for which | | must be less than U(PPTMR). If the number of results meeting this criterion is greater than or equal to the minimum, then the results are consistent with the expectations of the multivariate model, and the analyzer passes local validation. Detailed calculations are described in Section 11 and Annex A4.
1.4.1.4 Users must confirm that results aligning with the expectations based on the multivariate model are sufficient for the intended application. A 95% probability is recommended for the inverse binomial distribution calculation, though users may adjust this based on the criticality of the application. For further details, see Annex A4.
