Speaker
Description
When developing short forms of questionnaires, the most common approach under classical test theory is to retain the items with the highest factor loadings on each factor. Although this strategy generally produces short forms that perform well on standard reliability and validity metrics, it can unduly narrow the construct being measured by favouring highly correlated, homogenous items.
This study evaluates an alternative method: the genetic algorithm proposed by Tal Yarkoni (2010), a simple machine learning–based approach to item selection. We tested its performance using both simulated data and real-world data across three different types of scales: a highly redundant unifactorial scale, a less redundant unifactorial scale, and a three-factor scale with low item content overlap. Evaluation criteria included correlation between short and full scale scores, internal consistency (Cronbach’s alpha), and preservation of correlations with related constructs.
Compared to the traditional highest-loading method, the genetic algorithm yielded short forms with stronger correlations with full-scale scores (.93–.99 vs. .89–.98) and better preservation of construct validity, as indicated by smaller shifts in correlations with related constructs (standard deviation of differences: .014–.038 vs. .020–.096). However, the genetic algorithm produced slightly lower internal consistency (.74–.84 vs. .77–.92), due to its tendency to select more diverse items.
Overall, results suggest that the genetic algorithm represents a computationally efficient method to develop short forms that better retain the conceptual scope and external validity of the original instrument.
| If you're submitting a symposium talk, what's the symposium title? | New perspectives for developing short forms of tests |
|---|---|
| If you're submitting a symposium, or a talk that is part of a symposium, is this a junior symposium? | No |
