Technical Program

Paper Detail

Paper: PS-1B.9
Session: Poster Session 1B
Location: Symphony/Overture
Session Time: Thursday, September 6, 18:45 - 20:45
Presentation Time:Thursday, September 6, 18:45 - 20:45
Presentation: Poster
Paper Title: Performance Optimization is Insufficient for Building Accurate Models for Neural Representation
Manuscript:  Click here to view manuscript
Authors: Jonathan Yu, Qihong Lu, Uri Hasson, Kenneth Norman, Jonathan Pillow, Princeton University, United States
Abstract: Convolutional neural networks, optimized for image classification, are state-of-the-art computational models for visual neural representation. Moreover, performance optimization often leads to better models of neural representation (Yamins et al., 2014). In this study, we investigate whether performance optimization always increases the similarity between the neural network and the brain, in terms of their representations. We compared AlexNet and a residual network on a recent human image-viewing fMRI dataset (Horikawa & Kamitani, 2017). The original study found a remarkable similarity between AlexNet and the brain (Horikawa & Kamitani, 2017). Although residual networks achieved better image classification performance, we found that the hidden representation of the residual network is much less similar to human brain representation, compared to AlexNet. This result suggests that performance optimization can eventually lead to systematic deviation from human brain representation. We conclude that additional neuroscience-inspired design is critical for building a better representation model of the brain.