ENHANCING DANGEROUS OBJECT DETECTION THROUGH INTERMEDIATE MULTIMODAL DATA FUSION

Abstract

Reliable detection of dangerous objects is a critical component in numerous domains, including autonomous transportation, industrial safety, surveillance systems, and robotics. In real-world scenarios – such as poor lighting, adverse weather, occluded views, or noisy backgrounds – single-sensor detection systems often struggle to maintain accuracy and stability. To address these limitations, this paper proposes an approach based on multimodal data fusion, leveraging attention mechanisms and object criticality assessment.The proposed system integrates information from an RGB camera, LiDAR, thermal imager, and optionally, tactile sensors. Each sensor stream is processed by a dedicated feature encoder. Attention weights are dynamically calculated based on the environmental context to determine the relevance of each modality. The resulting fused feature vector is passed through a detection module that produces a base confidence score, while a parallel criticality module evaluates spatial distance, relative velocity, and orientation of the detected object.The final decision is formed by combining both the detection score and the criticality assessment using a weighted factor. If the resulting Sfinal value exceeds a predefined threshold, the system triggers a response – issuing a warning, initiating emergency braking, or notifying the operator. A hypothetical case study involving pedestrian detection in limited visibility illustrates the practical utility of the proposed architecture.The system exhibits high accuracy, robustness, and contextual adaptability. Its main advantages include preservation of spatial and semantic features, the ability to process inputs across multiple sensing domains, and risk-based prioritization of threats. This architecture presents a strong foundation for the development of reliable, safety-oriented autonomous solutions in transportation, industrial monitoring, and urban environments.