Reliability Analysis

Written by Tara Peris
Bookmark and Share

Reliability analysis is a term that is typically used in statistical circles to refer to the process of examining inter-rater agreement. However, the actual procedures that are employed may be used for much broader purposes. Of particular interest is the use of reliability analysis for product development purposes.

Traditionally, reliability analysis has been utilized to examine the extent to which to people agree on particular ratings. The kappa statistic is used to consider base-rates of various response options relative to the percentage of inter-rater agreement. Most often, researchers are required to present a kappa statistic rather than simply state the percentage of time that people reported the same rating, which higher kappas representing greater agreement.

Reliability Analysis and Product Development
However, reliability analysis has many other purposes as well. In particular, it may be used to evaluate a product's reliability. The fundamental idea is the same, although the statistics may vary a bit. This branch of statistics is perhaps best viewed through the lens of DFSS and Six Sigma engineering, approaches that seek to optimize the reliability of a given product.

Because of its many uses, it pays to learn a bit about reliability analysis and about conventions within your particular field. For example, within psychology, raters must maintain reliability ratings of at least 80%, a criterion that may be painfully unacceptable in other disciplines. Learn the basics of reliability analysis both as a way to enhance your own work and a way to interpret that of others.

Bookmark and Share