Abstract
Disaster management teams play a crucial role in responding to catastrophic events with speed and efficiency. However, when faced with large data of disaster-related information, manual systems can struggle to classify the information accurately, especially when they are unavailable. This challenge highlights the need for integrating social media and implementing machine learning models to address the issue. However, the development of such models is dependent on the availability of adequately annotated data, which presents a significant obstacle in the field of crisis management. to address this challenge, our study focuses on the need for disaster event classification through social media. Thus, we present a new technique called TweetACE, a fine-grained disaster tweet annotation technique that utilizes tweets from two disaster events. Our dual annotation approach assigns two labels to each tweet, reducing ambiguity and improving model training efficiency. the high-quality data annotation achieved through our rigorous post-Annotation processes is evident in the refined labels and impressive Krippendorff's alpha of 0.87 for event types and 0.84 for sub-event types, which indicate the reliability of annotation agreement between annotators. Furthermore, the preprocessing of textual data eliminates extraneous elements that contain noise, which can hinder the accuracy of classification models. This step enables us to refine the dataset and spotlight meaningful tweet content, ultimately boosting model accuracy. Our benchmark model, which uses the Bidirectional Encoder Representations from Transformers (BERT) model, achieved 66% and 57% accuracy for the event and sub-event categories, respectively, highlighting potential avenues for enhancement. in conclusion, our work presents a promising solution for disaster event classification through social media that could significantly enhance the effectiveness of disaster management teams. by providing a well-Annotated dataset for machine learning models to utilize, we can improve the accuracy of classification and ultimately aid disaster management teams in their critical work. Our dataset is available on GitHub1.
Recommended Citation
A. Adesokan et al., "Tweetace: A Fine-Grained Classification of Disaster Tweets using Transformer Model," Proceedings - Applied Imagery Pattern Recognition Workshop, Institute of Electrical and Electronics Engineers, Jan 2023.
The definitive version is available at https://doi.org/10.1109/AIPR60534.2023.10440656
Department(s)
Computer Science
International Standard Serial Number (ISSN)
2164-2516
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2025 Institute of Electrical and Electronics Engineers, All rights reserved.
Publication Date
01 Jan 2023
