Abstract

Is intelligence realized by connectionist or classicist? While connectionist approaches have achieved superhuman performance, there has been growing evidence that such task-specific superiority is particularly fragile in systematic generalization. This observation lies in the central debate between connectionist and classicist, wherein the latter continually advocates an algebraic treatment in cognitive architectures. In this work, we follow the classicist's call and propose a hybrid approach to improve systematic generalization in reasoning. Specifically, we showcase a prototype with algebraic representation for the abstract spatial-temporal reasoning task of Raven's Progressive Matrices (RPM) and present the ALgebra-Aware Neuro-Semi-Symbolic (ALANS) learner. The ALANS learner is motivated by abstract algebra and the representation theory. It consists of a neural visual perception frontend and an algebraic abstract reasoning backend: the frontend summarizes the visual information from object-based representation, while the backend transforms it into an algebraic structure and induces the hidden operator on the fly. The induced operator is later executed to predict the answer's representation, and the choice most similar to the prediction is selected as the solution. Extensive experiments show that by incorporating an algebraic treatment, the ALANS learner outperforms various pure connectionist models in domains requiring systematic generalization. We further show the generative nature of the learned algebraic representation; it can be decoded by isomorphism to generate an answer.

Paper

Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning
Chi Zhang*, Sirui Xie*, Baoxiong Jia*, Ying Nian Wu, Song-Chun Zhu, Yixin Zhu
Proceedings of the European Conference on Computer Vision (ECCV), 2022
(* indicates equal contribution.)
Paper / Supplementary / Code / Blog

Team

Chi Zhang1

Sirui Xie1

Baoxiong Jia1

Ying Nian Wu1

Song-Chun Zhu1,2,3,4

Yixin Zhu2

1 University of California, Los Angeles

2 Peking University

3 Tsinghua University

4 Beijing Institute for General Artificial Intelligence (BIGAI)

Dataset

The new splits introduced in this work follow the same design in the original work of RAVEN and I-RAVEN, except that the training set and the test set are sampled from different distributions of relations. All the splits are of the 3x3 grid layout. See the supplementary material for details.

Download the dataset here. The zip file contains six folders (Systematicity, Productivity, and Localism for the two data-generating methods). Those starting with "I" are generated from the I-RAVEN method.

Code

View on GitHub

Bibtex

@inproceedings{zhang2022learning,
 title={Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning},
 author={Zhang, Chi and Xie, Sirui and Jia, Baoxiong and Wu, Ying Nian and Zhu, Song-Chun and Zhu, Yixin},
 booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
 year={2022}
}