Speaker
Description
When planning an experiment, the number of participants is typically based on the desired statistical power or precision. For intrinsically noisy measures such as reaction times, neural or physiological indexes, the number of trials per participant can also be increased to enhance statistical power. However, the actual benefit of adding more trials has not been systematically evaluated. Longer experiments may introduce fatigue, reduce measurement precision, or cause adaptation and ceiling effects, potentially hindering statistical Power. Using three classic psychological paradigms (Simon effect, SNARC effect, Task Switching), we descriptively examine how the number of trials affects effect size, statistical power, and parameter estimates. Through a cumulative exploratory modelling approach, we show that increasing the number of trials can sometimes have no benefit — and may even be detrimental, reducing effect size and thus lowering power. We propose benchmarks for balancing the number of trials against effect size and precision. Even for other paradigms, we recommend evaluating experiment length alongside the usual power analysis, accounting for potential side effects.
