Speaker
Description
Humans and animals often engage in behaviors not driven by immediate extrinsic rewards, such as exploring novel objects. This tendency to be curious, conserved across species, is thought to depend on intrinsic motivation, described in psychology as a general-purpose mechanism supporting adaptation to changing environments. Meanwhile, the AI community has incorporated intrinsic motivation into policies for artificial agents, showing performance improvement in specific tasks. However, despite these parallel efforts, a unified framework explaining how intrinsic motivation and curiosity may emerge from an elementary drive towards survival (or as a generalization over useful extrinsic goals) is still lacking.
We hypothesize that curiosity arises through repeated interactions with dynamic environments featuring variable action-reward contingencies, both within and across generations. To test this, we trained artificial agents on a reward collection task in a grid world with two rooms connected by a corridor. The initial room contains an object and sparse rewards, while the second room is richer but initially inaccessible. Although the object itself provides no reward, visiting it unlocks access to the richer room. Crucially, object identity and position vary across trials, requiring agents to abstract the relevance of novelty beyond trial-specific features. We show that recurrent neural networks (RNNs) learn the task and develop a preference for the object, suggesting the emergence of an intrinsic preference for an unrewarding item.
Importantly, novelty is a relative property: an object is novel only with respect to previously encountered items. To make the emerging tendency more comparable to biological curiosity towards novel objects, we are currently testing RNNs in a task variant where the target object must be identified among non-instrumental distractors. Finally, we aim to translate this paradigm into a human experiment, enabling direct comparison with artificial agents within a unified framework for studying intrinsic motivation and curious behavior.
| Preferred Presentation | Oral Presentation |
|---|