Multi-Scale Sparse Network with Cross-Attention Mechanism for Image-Based Butterflies Fine-Grained Classification


Butterfly protection is critical for environmental protection, and butterfly classification study is an essential tool for doing so. We proposed a new fine-grained butterfly classification architecture to address the issues of duplicate information in some butterfly images and trouble identifying them due to their tiny inter-class variance. To begin, a Non-Local Mean Filtering and Multi-Scale Retinex-based method (NL-MSR) is employed to enhance the butterfly images in order to efficiently retain more detail information. Then, to accomplish fine-grained categorization of butterfly images, a Multi-scale Sparse Network with Cross-Attention Mechanism (CA-MSNet) is designed. In CA-MSNet, a Cross-Attention Mechanism module (CAM) that offers distinct weights in the horizontal and vertical directions based on two strategies is devised to successfully identify the spatial distribution of butterfly stripes and spots and suppress incorrect information. Then, to overcome the recognition problem of butterfly spots with small inter-class variance, a Multi-scale sparse module (MSS) with multi-scale receptive fields is constructed. Finally, a Depthwise Separable Convolution module is employed to mitigate the parameter rise induced by the MSS network. In order to validate the model's feasibility and effectiveness in a complex environment, we compared it to existing methods, and our proposed method achieved an average recognition accuracy of 91.88%, with an F1 value of 92.15%, indicating that it has a good effect on the fine-grained classification of butterflies and can be applied to their recognition to realize their protection.


Civil, Architectural and Environmental Engineering


This work was supported by the National Natural Science Foundation of China (Grant No. 61703441); in part by Changsha Municipal Natural Science Foundation (Grant No. kq2014160); in part by the Natural Science Foundation of Hunan Province (Grant No. 2021JJ41087); in part by Hunan Key Laboratory of Intelligent Logistics Technology (2019TP1015).

Keywords and Phrases

Butterfly Fine-Grained Classification; Cross-Attention Mechanism; Depthwise Separable Convolution Module; Image-Based Butterflies; Multi-Scale Sparse Structure; NL-MSR Image Enhancement

International Standard Serial Number (ISSN)


Document Type

Article - Journal

Document Version


File Type





© 2022 Elsevier, All rights reserved.

Publication Date

01 Mar 2022