Learning to See Lesions, Not Skin Tone: Counterfactual Multimodal Learning for Fair, Trustworthy, and Text-Free Dermatology AI

Abstract

Recent advances in deep learning have significantly improved the performance of automated skin lesion classification systems, enabling accurate detection of various dermatological conditions from medical images. Despite these achievements, concerns regarding fairness and generalization remain a major challenge for the deployment of such systems in real-world clinical settings. A key factor contributing to these challenges is the presence of bias in training datasets, particularly with respect to skin-tone representation. Most publicly available skin lesion datasets contain a disproportionate number of samples from individuals with lighter skin tones. As a result, deep learning models trained on these datasets often learn representations that perform well for majority populations while exhibiting degraded performance on underrepresented groups. Such disparities can lead to unequal diagnostic outcomes and raise important concerns regarding the reliability and fairness of artificial intelligence systems in healthcare. The problem is especially relevant in the Indian context, where substantial diversity exists in skin tone, ethnicity, and geographical distribution. Variations in lesion appearance across di!erent skin types, combined with the limited availability of representative datasets, make it di”cult to develop models that generalize e!ectively to the broader population. Consequently, addressing dataset bias and improving model robustness across diverse demographic groups has become an important research objective in skin lesion analysis. Motivated by these challenges, this dissertation investigates fairness-aware approaches for skin lesion classification. The work focuses on the creation and analysis of diverse skin lesion datasets, the study of bias mitigation techniques, and the development of robust deep learning models capable of learning equitable representations. Furthermore, the dissertation explores invariant and equivariant learning paradigms as potential mechanisms for improving generalization across heterogeneous data distributions. To enhance model transparency and support trustworthy decision-making, explainability techniques are also examined to provide insights into the features utilized by the learned models during classification. Through these investigations, the dissertation aims to contribute toward the development of more reliable, interpretable, and fair deep learning systems for skin lesion analysis, with particular emphasis on improving performance across diverse skin-tone distributions.