Sep 11 – 13, 2025
Campus Luigi Einaudi
Europe/Rome timezone

TOWARDS A NEW MEASURE OF COGNITIVE ABSTRACTION: LLM-ASSISTED DEVELOPMENT OF A NEW PSYCHOMETRIC INSTRUMENT

Sep 12, 2025, 11:45 AM
15m
Aula A1 (Pareto)

Aula A1 (Pareto)

Speaker

Margherita Attanasio (Università degli Studi dell'Aquila)

Description

Introduction
Abstract intelligence and non-verbal thinking are essential components in understanding cognitive potential, especially in individuals with intellectual giftedness. However, traditional psychometric measures tend to underestimate these dimensions, favouring verbal and academic components. This study proposes the development of a new instrument for assessing cognitive abstraction, based on symbolic, logical, visuospatial and divergent items. Particularly, the study explores the usefulness of Large Language Models (LLM) in the assisted construction of psychometric tests, exploiting their generative capacity to develop content consistent with theoretical models of intelligence and adaptable to different cognitive profiles.

Method
The instrument consists of 70 items divided into subsections (visual matrices, mental rotations, logical proportions, divergent tasks). The LLM was used to generate, adapt and organise the items, ensuring semantic coherence and difficulty progression. The battery was administered to 100 individuals, aged between 18 and 40 years. The Verbal Judgements Test was used as a comparison measure.

Results and Conclusions
The psychometric analyses conducted on the items, with particular focus on Item Response Theory, together with the evidence of validity and reliability, confirm the methodological robustness of the instrument and its adequacy in measuring the construct under investigation. The use of the LLM made it possible to design an innovative, multidimensional and culturally fair psychometric instrument. The model offers a new paradigm of test design, in which generative artificial intelligence becomes a concrete resource for psychological research. The results open promising perspectives for the use of AI in the construction of flexible, sensitive and scientifically based assessment instruments.

If you're submitting a symposium talk, what's the symposium title? Menti artificiali e menti umane: l'intelligenza al confine tra psicologia e tecnologia
If you're submitting a symposium, or a talk that is part of a symposium, is this a junior symposium? No

Primary authors

Monica Mazza (Università degli Studi dell'Aquila) Margherita Attanasio (Università degli Studi dell'Aquila)

Presentation materials

There are no materials yet.