Technical Program

Paper Detail

Paper: PS-1A.24
Session: Poster Session 1A
Location: Symphony/Overture
Session Time: Thursday, September 6, 16:30 - 18:30
Presentation Time:Thursday, September 6, 16:30 - 18:30
Presentation: Poster
Paper Title: Noisy inference of value signals in frontal cortex drives exploration during reward-guided learning
Manuscript:  Click here to view manuscript
Authors: Charles Findling, Vasilisa Skvortsova, Ecole normale supérieure, France; Rémi Dromnelle, Université Pierre et Marie Curie, France; Nicolas Chopin, École nationale de la statistique et de l'administration économique, France; Stefano Palminteri, Valentin Wyart, Ecole normale supérieure, France
Abstract: When tracking rewarded stimulus-response associations in volatile environments, humans make a surprisingly large number of seemingly suboptimal decisions, which do not maximize expected outcome. These ‘exploratory’ decisions have been assigned either to information seeking or to stochasticity in response selection. We reasoned that a fraction of exploratory decisions could be due to random noise in the inference process driving learning, noise which is otherwise assumed to be negligible. Accounting simultaneously for these different sources of exploration in reinforcement learning revealed that more than half of exploratory decisions are due to inference noise alone. This computational dissection of exploration is supported by neuroimaging data, which shows a dissociation in the relationship between choice behavior and two brain regions associated with exploration: fluctuations in anterior cingulate activity co-vary with inference noise during learning, whereas frontopolar activity drives exploration during choice. Together, these findings indicate that exploration in reward-guided learning is driven to a large part by random errors in inference, unbeknownst to the decision-maker.