Deep Learning and Handcrafted Method Fusion: Higher Diagnostic Accuracy for Melanoma Dermoscopy Images
Abstract
This paper presents an approach that combines conventional image processing with deep learning by fusing the features from the individual techniques. We hypothesize that the two techniques, with different error profiles, are synergistic. The conventional image processing arm uses three handcrafted biologically inspired image processing modules and one clinical information module. The image processing modules detect lesion features comparable to clinical dermoscopy information - atypical pigment network, color distribution, and blood vessels. The clinical module includes information submitted to the pathologist - patient age, gender, lesion location, size, and patient history. The deep learning arm utilizes knowledge transfer via a ResNet-50 network that is repurposed to predict the probability of melanoma classification. The classification scores of each individual module from both processing arms are then ensembled utilizing logistic regression to predict an overall melanoma probability. Using cross-validated results of melanoma classification measured by area under the receiver operator characteristic curve (AUC), classification accuracy of 0.94 was obtained for the fusion technique. In comparison, the ResNet-50 deep learning based classifier alone yields an AUC of 0.87 and conventional image processing based classifier yields an AUC of 0.90. Further study of fusion of conventional image processing techniques and deep learning is warranted.
Recommended Citation
J. R. Hagerty and R. J. Stanley and H. A. Almubarak and N. Lama and R. Kasmi and P. Guo and R. J. Drugge and H. S. Rabinovitz and M. Oliviero and W. V. Stoecker, "Deep Learning and Handcrafted Method Fusion: Higher Diagnostic Accuracy for Melanoma Dermoscopy Images," IEEE Journal of Biomedical and Health Informatics, vol. 23, no. 4, pp. 1385 - 1391, Institute of Electrical and Electronics Engineers (IEEE), Jul 2019.
The definitive version is available at https://doi.org/10.1109/JBHI.2019.2891049
Department(s)
Electrical and Computer Engineering
Second Department
Chemistry
Keywords and Phrases
Blood vessels; Classifiers; Dermatology; Diagnosis; Image fusion; Image processing; Knowledge management; Oncology; Biologically inspired; Classification accuracy; Dermoscopy; Image Processing Module; Image processing technique; Melanoma; Receiver operator characteristic curves; Transfer learning; Deep learning
International Standard Serial Number (ISSN)
2168-2194
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2019 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
01 Jul 2019
Comments
This work was supported in part by the National Institutes of Health under Grants SBIR R43 CA153927-01 and CA101639-02A2.