The goal of this study was to investigate the specific patterns of memory breakdown in patients suffering from early-onset Alzheimer's disease (EOAD) and late-onset Alzheimer's disease (LOAD). Twenty EOAD patients, twenty LOAD patients, twenty matched younger controls, and twenty matched older controls participated in this study. All participants underwent a detailed neuropsychological assessment, an MRI scan, an FDG-PET scan, and AD patients had biomarkers as supporting evidence of both amyloïdopathy and neuronal injury. Results of the neuropsychological assessment showed that both EOAD and LOAD groups were impaired in the domains of memory, executive functions, language, praxis, and visuoconstructional abilities, when compared to their respective control groups. EOAD and LOAD groups, however, showed distinct patterns of memory impairment. Even though both groups were similarly affected on measures of episodic, short term and working memory, in contrast semantic memory was significantly more impaired in LOAD than in EOAD patients. The EOAD group was not more affected than the LOAD group in any memory domain. EOAD patients, however, showed significantly poorer performance in other cognitive domains including executive functions and visuoconstructional abilities. A more detailed analysis of the pattern of semantic memory performance among patient groups revealed that the LOAD was more profoundly impaired, in tasks of both spontaneous recall and semantic recognition. Voxel-Based Morphometry (VBM) analyses showed that impaired semantic performance in patients was associated with reduced gray matter volume in the anterior temporal lobe (ATL) region, while PET-FDG analyses revealed that poorer semantic performance was associated with greater hypometabolism in the left temporoparietal region, both areas reflecting key regions of the semantic network. Results of this study indicate that EOAD and LOAD patients present with distinct patterns of memory impairment, and that a genuine semantic impairment may represent one of the clinical hallmarks of LOAD.