Analysis of measurement precision experiment with categorical variables

Evaluating performance of a measurement method is essential in metrology. Concepts of repeatability and reproducibility are introduced in ISO5725-1 (1994) including how to run and analyse experiments (usually collaborative studies) to obtain these precision measures. ISO5725-2 (1994) describe precision evaluation in quantitative measurements but not in qualitative measurements. Some methods have been proposed for qualitative measurements cases such as Wilrich (2010), de Mast & van Wieringen (2010), Bashkansky, Gadrich & Kuselman (2012). Item response theory (Muraki, 1992) is another methodology that can be used to analyse qualitative data. Utilizing these methods, analysis of measurement precision experiment is investigated when the measurements involve categorical variables. The data analysed are from the precision experiment of intratracheal administration testing (AIST, 2018) whose objectives were to study the precision of the standardized test method for evaluating the pulmonary toxicity of nanomaterials. In such experiments, dose-response relationship also need to be considered which makes the situation more complicated. We discuss how these data should be analysed using actual data.