During consultation dermatologists have to address hundreds of lesions in a limited amount of time. They will not only evaluate the single lesion of interest but more importantly the context of it. Visually comparing the similarity of the majority of lesions within the same patient provides a strong indication for lesions with significantly differing aspects. Deep learning algorithms are capable to identify such outliers, i.e. images that differ considerably from the expected appearance on a larger cohort, and highlight the main differences in those cases. In the present study we evaluate the use of autoencoders as unsupervised tools to detect suspicious skin lesions based on evaluation of real world data acquired during consultation at the USZ Dermatology Clinic. Clinical Relevance- Deep learning algorithms are showing many promising results in dermatology lesion classification. However the context of the lesion is normally not considered in the analysis which prevents these tools to transition into routine practice. An outlier detector based on real world data would allow a dermatologist or general practitioner to detect the suspicious lesions for further examination. The algorithm would additionally provide useful insights by highlighting the feature differences between the original outlier (malignant lesion) and the lesion reconstructed by the autoencoder.