In the present article, I question the claim that a literacy bias is responsible for the fact that major theoreticians underestimated or ignored the role of literacy in spoken language. Instead, I argue that the strongly modular, localist and symbolic information-processing approach to cognition that has dominated psychological science throughout the 20th century has prevented cross-fertilization and the emergence of a unified theory of written and spoken language processing. I show that the recognition of the fundamental role of phonology in reading had suffered from exactly the same “bias”, which had eventually been overcome not by breaking the “literacy glasses” but by shifting theoretical frameworks. I conclude by arguing that a marriage between cognitive science, evolutionary biology and neuroscience is needed more than ever to develop a unified, developmentally, biologically and evolutionary plausible theory of written and spoken language.