Melanoma, a highly aggressive form of skin cancer notorious for its rapid metastasis, necessitates early detection to mitigate complex treatment requirements. While considerable research has addressed melanoma diagnosis using convolutional neural networks (CNNs) on individual dermatological images, a deeper exploration of lesion comparison within a patient is warranted for enhanced anomaly detection, which often signifies malignancy. In this study, we present a novel approach founded on an automated, self-supervised framework for comparing skin lesions, working entirely without access to ground truth labels. Our methodology involves encoding lesion images into feature vectors using a state-of-the-art representation learner, and subsequently leveraging an anomaly detection algorithm to identify atypical lesions. Remarkably, our model achieves robust anomaly detection performance on ISIC 2020 without needing annotations, highlighting the efficacy of the representation learner in discerning salient image features. These findings pave the way for future research endeavors aimed at developing better predictive models as well as interpretable tools that enhance dermatologists' efficacy in scrutinizing skin lesions.