Explaining the Unseen: Multimodal Vision-Language Reasoning for Situational Awareness in Underground Mining Disasters

Author

• Mizanur Rahman Jewel
• Mohamed Elmahallawy
• Sanjay Kumar Madria, Missouri University of Science and TechnologyFollow
• Samuel Frimpong, Missouri University of Science and TechnologyFollow

Abstract

Underground mining disasters produce pervasive darkness, dust, and collapses that obscure vision and make situational awareness difficult for humans and conventional systems. To address this, we propose MDSE, Multimodal Disaster Situation Explainer, a novel vision-language framework that automatically generates detailed textual explanations of post-disaster underground scenes. MDSE has three-fold innovations: (i) Context-Aware Cross-Attention for robust alignment of visual and textual features even under severe degradation; (ii) Segmentation-aware dual pathway visual encoding that fuses global and region-specific embeddings; and (iii) Resource-Efficient Transformer-Based Language Model for expressive caption generation with minimal compute cost. To support this task, we present the Underground Mine Disaster (UMD) dataset - the first image-caption corpus of real underground disaster scenes - enabling rigorous training and evaluation. Extensive experiments on UMD and related benchmarks show that MDSE substantially outperforms state-of-the-art captioning models, producing more accurate and contextually relevant descriptions that capture crucial details in obscured environments, improving situational awareness for underground emergency response. The code is at Github.

Recommended Citation

M. R. Jewel et al., "Explaining the Unseen: Multimodal Vision-Language Reasoning for Situational Awareness in Underground Mining Disasters," Proceedings 2026 IEEE Cvf Winter Conference on Applications of Computer Vision Wacv 2026, pp. 1324 - 1333, Institute of Electrical and Electronics Engineers, Jan 2026.

The definitive version is available at https://doi.org/10.1109/WACV61042.2026.00135

Department(s)

Computer Science

Second Department

Mining Engineering

Comments

Centers for Disease Control and Prevention, Grant None

Document Type

Article - Conference proceedings

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2026 Institute of Electrical and Electronics Engineers, All rights reserved.

Publication Date

01 Jan 2026