Speaker
Description
How do we understand each other? Inferring communicative intentions – such as clarification requests, invitations, orders, statements – is fundamental to social interaction. Without this ability, a question like "Can you pass me the salt?" would be met with a literal "Yes, I can" rather than the intended action.
Crucially, in face-to-face interaction, inferring intentions relies not only on linguistic meaning, but is also fundamentally shaped by prosodic intonation and visual cues such as facial expressions and head movements. How prosodic and facial cues contribute to this process, however, remains largely unexplored. Here, we aim to systematically assess how the two jointly shape intention recognition.
In a categorization task, participants will judge the intention (clarification, statement, invitation, order, other) expressed by an actor who utters a single word in audiovisual recordings. Normed single words were chosen to reduce their semantic influence on intention recognition.
A perturbation paradigm recombines audio and video across intentions, varying how plausible each pairing is in natural communication.
We predict that intention arises from the flexible arbitration between cue integration and segregation, formalized by Bayesian causal inference: cues attributed to a common source fuse into non-additive percepts (a combined reading different from the sum of the single cues), while cues judged independent yield reliance on the most diagnostic one.
Further, fMRI and EEG experiments will examine how this process is implemented across the cortical hierarchy. Representational Similarity Analysis (RSA) will test where (fMRI) and when (EEG) neural representational dissimilarity matrices (RDMs) align with model-derived RDMs at multiple processing levels, from low-level sensory features to higher-level representations (confusion matrices from behavioral performance and computational modelling).
Collectively, the project will unveil the computational principles and neural implementation driving how the brain recovers communicative intentions from face-to-face interaction.
| Preferred Presentation | Poster Presentation |
|---|