Understanding the complexity of computational models through optimization and sloppy parameter analyses: The case of the Connectionist Dual-Process Model - Aix-Marseille Université
Journal Articles Journal of Memory and Language Year : 2024

Understanding the complexity of computational models through optimization and sloppy parameter analyses: The case of the Connectionist Dual-Process Model

Conrad Perry
  • Function : Author
  • PersonId : 1289799
Rick Evertz
  • Function : Author
Marco Zorzi
  • Function : Author

Abstract

A major strength of computational cognitive models is their capacity to accurately predict empirical data. However, challenges in understanding how complex models work and the risk of overfitting have often been addressed by trading off predictive accuracy with model simplification. Here, we introduce state-of-the-art model analysis techniques to show how a large number of parameters in a cognitive model can be reduced into a smaller set that is simpler to understand and can be used to make more constrained predictions with. As a test case, we created different versions of the Connectionist Dual-Process model (CDP) of reading aloud whose parameters were optimized on seven different databases. The results showed that CDP was not overfit and could predict a large amount of variance across those databases. Indeed, the quantitative performance of CDP was higher than that of previous models in this area. Moreover, sloppy parameter analysis, a mathematical technique used to quantify the effects of different parameters on model performance, revealed that many of the parameters in CDP have very little effect on its performance. This shows that the dynamics of CDP are much simpler than its relatively large number of parameters might suggest. Overall, our study shows that cognitive models with large numbers of parameters do not necessarily overfit the empirical data and that understanding the behavior of complex models is more tractable using appropriate mathematical tools. The same techniques could be applied to many different complex cognitive models whenever appropriate datasets for model optimization exist.
Fichier principal
Vignette du fichier
2024_Sloppy_Parameters_JML2.pdf (3.62 Mo) Télécharger le fichier
Origin Publication funded by an institution
Licence

Dates and versions

hal-04226995 , version 1 (03-10-2023)

Licence

Identifiers

Cite

Conrad Perry, Rick Evertz, Marco Zorzi, Johannes C. Ziegler. Understanding the complexity of computational models through optimization and sloppy parameter analyses: The case of the Connectionist Dual-Process Model. Journal of Memory and Language, 2024, 134, pp.104468. ⟨10.1016/j.jml.2023.104468⟩. ⟨hal-04226995⟩
93 View
80 Download

Altmetric

Share

More