Abstract

Dermatology skin lesion image analysis research has been ongoing at Missouri S&T (previously UMR) since the 1980s. Our group has been successful in finding over 20 key dermoscopic structures in melanoma, melanoma mimics, and nonmelanoma skin cancers using iterative structure-based analysis. Structures are chosen to reduce system errors which are concentrated in a few classes: amelanotic/featureless, regressed, and small in situ melanomas, and the most difficult benign lesions to identify: Clark nevi, lentigines, and seborrheic keratoses [1]. Preliminary research detecting and using annotated lesion structures [2-7] and lesion artifacts [8,9] guided by clinical experience [10] with image processing and deep learning-based architectures for segmentation and lesion classification. Structures that computer vision can detect allow more accurate diagnosis of challenging lesions. We have used inspectable structure segmentation deep learning outputs with associated lesion classification to facilitate the correlation of structures with misdiagnosed lesions; this gives a path to exploring the relationship between expert annotated structures with automated segmentation and lesion classification.

Our current research focuses on multi-expert annotation of key skin lesion structures from small-scale datasets with computational learning and fusion with existing computational whole-lesion methods to enhance automated structure detection and lesion discrimination capability. Data validation is an important component of identifying key structures in skin lesions due to inter-expert variability. Deep learning techniques have, up to now, been applied to whole dermoscopy images. Processing at the whole-lesion (global) scale may miss critical structures. Resulting errors may be overcome by detecting critical structures in a lesion. Curating small-scale datasets of annotated key structures associated with specific lesion types by multiple experts provides two significant, novel contributions: 1) validation of key structures used for lesion diagnosis that are disseminated in the research community and 2) fusion of expert and machine learning in a reinforcement learning framework to promote enhanced lesion classification and interpretability of machine learning classification decisions.

The International Skin Imaging Collaboration (ISIC) archive offers the largest publicly available collection of dermoscopic images for skin lesion analysis. However, the ISIC 2019 dataset suffers from several acquisition artifacts such as hair occlusions, vignetting, surgical ink markings, and ruler scales, all of which compromise its utility in deep learning workflows by introducing non-biological visual structures. A rigorously annotated artifact archive of ISIC 2019 is given here that includes binary masks for key artifact types, including ink marks, vignetting, hair and ruler marks. The annotations were obtained through manual (ink marks and vignetting) and semi-automatic (hairs and ruler marks) systematic review process. Separate .zip files with the annotation masks for the different artifact types are provided for the ISIC 2019 dataset. These artifact archive annotation collections provide an initial step to develop a multi-expert data annotation process with quality control that can be extended to dataset annotation of other critical skin lesion features for expert-in-the-loop, deep learning whole-lesion algorithm development, and data fusion for enhanced skin lesion analysis.

Viewing Instructions

The ISIC 2019 original image dataset with is accessible at https://challenge.isic-archive.com/data/#2019 [11-13]. The annotated mask images for hair and ruler marks, ink marks, and vignetting for the dermoscopy skin lesion images in the ISIC 2019 dataset are contained in separate .zip files for each artifact. The .zip files for each artifact that can be downloaded below in the supplemental files.

The naming convention used for the annotations in each .zip file is:

_.tiff

Where ISIC_2019_filename references the original filename in the ISIC 2019 dataset that the annotation corresponds to. The ANNOTATION_TYPE one of the following types:

  • mvig (vignetting masks)
  • hairmask (hair and ruler masks)
  • inkmark (ink mark masks)

Each annotation is saved as a 1-channel binary image with the same image resolution as its corresponding ISIC 2019 data sample. The grayscale value of 255 represents the regions of interest for each annotation, else a value of 0 for all other areas.

Sponsor(s)

This research is supported by the NIH grant (1R15CA287304-01) entitled “Skin Lesion Archive and Annotation for Fusion and Earlier Skin Cancer Detection.”

Comments

References:

[1] Ferris LK, Harkes JA, Gilbert B, Winger DG, Golubets K, Akilov O, Satyanarayanan M. Computer-aided classification of melanocytic lesions using dermoscopic images. J Am Acad Dermatol. 2015 Nov;73(5):769-76. PMID: 26386631.

[2] Maurya A, Stanley RJ, Lama N, Jagannathan S, Saeed D, Swinfard S, Hagerty JR, Stoecker WV. A deep learning approach to detect blood vessels in basal cell carcinoma. Skin Research and Technology, vol. 28, no. 4, pp. 571-576, 2022. Doi: 10.1111/srt.13150

[3] Nambisan A, Lama N, Thanh P, Swinfard S, Lama B, Smith C, Rajeh A, Patel G, Hagerty J, Stoecker WV, Stanley RJ. Deep learning-based dot and globule segmentation with pixel and blob-based metrics for evaluation. Intelligent Systems and Applications, vol. 16, 2022. https://doi.org/10.1016/j.iswa.2022.200126

[4] Nambisan AK, Maurya A, Lama N, Phan T, Patel G, Miller K, Lama B, Hagerty J, Stanley R, Stoecker WV. Improving Automatic Melanoma Diagnosis Using Deep Learning-Based Segmentation of Irregular Networks. Cancers (Basel). 2023 Feb 16;15(4):1259. doi: 10.3390/cancers15041259. PMID: 36831599; PMCID: PMC9953766

[5] Maurya A, Stanley RJ, Aradhyula HY, Lama N, Nambisan A, Patel G, Saeed D, Swinfard S, Smith C, Jagannathan S, Hagerty J, Stoecker WV. Basal cell carcinoma diagnosis with fusion of deep learning and telangiectasia features. Journal of Digital Imaging, vol. 37, no. 3, pp. 1137-1150, 2024. doi: 10.1007/s10278-024-00969-3

[6] Nambisan A, Lama N, Thanh P, Swinfard S, Lama B, Smith C, Rajeh A, Patel G, Hagerty J, Stoecker WV, Stanley RJ. Deep learning-based dot and globule segmentation with pixel and blob-based metrics for evaluation. Intelligent Systems and Applications, vol. 16, 2022. https://doi.org/10.1016/j.iswa.2022.200126

[7] Stoecker WV, Gupta K, *Stanley RJ, Moss RH, Shrestha B. Detection of Asymmetric Blotches (Asymmetric Structureless Areas) in Dermoscopy Images of Malignant Melanoma Using Relative Color. Skin Research and Technology. 2005;11:179-184.

[8] Kasmi R, Hagerty J, Young R, Lama N, Nepal J, Miinch J, Stoecker W, Stanley RJ. SharpRazor: Automatic removal of hair and ruler marks from dermoscopy images. Skin Res Technol. 2023 Apr;29(4):e13203. doi: 10.1111/srt.13203. PMID: 37113095.

[9] Lama N, Kasmi R, Hagerty JR, Stanley RJ, Young R, Miinch J, Nepal J, Nambisan A, Stoecker WV. ChimeraNet: U-Net for Hair Detection in Dermoscopic Skin Lesion Images. Journal of Digital Imaging. 2022 Nov. DOI: 10.1007/s10278-022-00740-6. PMID: 36385676.

[10] Stoecker WV, Moss RH, Ercal F, Umbaugh SE. Nondermatoscopic digital imaging of pigmented lesions. Skin Res Technol. 1995 Feb;1(1):7-16. PMID: 27328215.

[11] Tschandl P., Rosendahl C. & Kittler H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 180161 doi.10.1038/sdata.2018.161 (2018).

[12] Noel C. F. Codella, David Gutman, M. Emre Celebi, Brian Helba, Michael A. Marchetti, Stephen W. Dusza, Aadi Kalloo, Konstantinos Liopyris, Nabin Mishra, Harald Kittler, Allan Halpern: "Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)", 2017; arXiv:1710.05006.

[13] Hernández-Pérez C, Combalia M, Podlipnik S, Codella NC, Rotemberg V, Halpern AC, Reiter O, Carrera C, Barreiro A, Helba B, Puig S, Vilaplana V, Malvehy J. BCN20000: Dermoscopic lesions in the wild. Scientific Data. 2024 Jun 17;11(1):641.

Document Type

Data

Document Version

Citation

File Format

text

Language(s)

English

Publication Date

2019

Download

Additional files available below

Share

 
COinS