Speaker
Description
When confronted with dynamic and naturalistic visual input, the brain is continuously always trying to predict what’s coming next at different temporal scales and levels of feature complexity. Prediction would require stimulus-related features to be represented before their actual occurrence. This is also reflected in the active decisions that animals make. For example, the position to fixate in order to “foresee” future information. This hypothesis can be tested by measuring the moment in time at which the neural manifold and/or behavior are most aligned with the visual input.
In our MEG experiment, humans are presented with the same movie (“1917”) twice. The movie is highly dynamic and perceptually rich. Participants could freely move their eyes across the screen. On top of the predictions that generally occur in naturalistic settings, we expect the second repetition to be anticipated more in advance with respect to the first. A dynamic extension of representational similarity analysis (dRSA) is applied to single out different levels of the predictive processing hierarchy and how they are modulated by stimulus familiarity. dRSA essentially estimates subjects’ average predictions or reactions to stimulus features, their strength and latency.
We observe a clear pattern when analyzing frame-wise visual saliency. Preliminary results show that the object-level is better predicted and more in advance with respect to lower-level visual saliency both for gaze behavior, occipital and parietal MEG sensors. As models for the neural data, we (will) use also convolutional neural networks, visual transformers and other artificial neural networks that capture different facets of visual information, like body posture or body motion. Finally, we will include a multimodal neural network that takes as input videos, instead of static images, to investigate the impact of temporal narrative on neural representations and gaze behavior.
