The Similarity Index: A simple deductive measure to evaluate the empirical adequacy of a theoretical construct
Authors: Adrian Stanciu (GESIS-Leibniz Institute for the Social Sciences, Mannheim; Jacobs University Bremen, Bremen), Erich H. Witte (Hamburg University, Hamburg), and Frank Zenker (Bogazici University, Istanbul)
Abstract: Formal measures of effect size (ES) quantify the impact of an intervention on a dependent variable of interest. These measures are also widely used today to evaluate the empirical adequacy of a theoretical construct by comparing a predicted ES to an observed ES. A standardized ES-measure such as Cohen’s d, however, evaluates the observed ES in view of the observed standard deviation. The evaluation thus comes to include setting-specific information that a theoretical construct does not model, and so cannot predict. A critical evaluation of the empirical adequacy of a theoretical construct should arguably compare the theoretical ES directly to the observed ES. To this end, the similarity index (ISIM) offers a simple deductive index, intended to complement the use of standardized ES-measure in evaluating a theoretical construct for empirical adequacy. We simulate the observed ES and the sample size needed for ISIM to identify a theoretical ES that promises to agree with observations. We also describe how to apply ISIM in different research designs, and analyze select data from social and personality psychology. Rather than capitalizing on data quality alone, we can, and should, become more precise in our theoretical predictions. This paper introduces a practical measure to be used in determining how empirically adequate a theoretical prediction is. Here, we emphasize a discussion around effect size coefficients–Cohen’s d–and how a quantifiable index of the similarities between theorized and observed effect sizes–ISIM–can contribute to theoretical development.