1
Anomalous Activity Detection Using Deep Learning Techniques in Autonomous Vehicles

Amit Juyal1,2, Sachin Sharma1 and Priya Matta1*

1Department of Computer Science and Engineering, Graphic Era (Deemed to be University), Dehradun, Uttarakhand, India

2School of Computing, Graphic Era Hill University, Dehradun, Uttarakhand, India

Abstract

Autonomous driving is self-driving without the intervention of a human driver. A self-driving autonomous vehicle is designed with the help of high-technology sensors that can sense the traffic and traffic signals in the surroundings and move accordingly. It becomes necessary for a self-driving vehicle to take a right decision at the right time in an uncertain traffic environment. Any unusual anomalous activity or unexpected obstacle that could not be detected by an autonomous vehicle can lead to a road accident. For decision making in autonomous vehicles, very precisely designed and optimized programming software are developed and intensively trained to install in vehicle’s computer system. But in spite of these trained software some of the anomalous activity could become a hindrance to detect promptly during self-driving. Therefore, automatic detection and recognition of anomalies in autonomous vehicles is critical to a safe drive. In this chapter we discuss and propos deep learning method for anonymous activity detection of other vehicles that can be danger for safe driving in an autonomous vehicle. The present chapter focuses on various conditions and possible anomalies that should be known to handle while developing software for autonomous vehicles using deep learning models. A variety of deep learning models were tested to detect abnormalities, and we discovered that deep learning models can detect anomalies in real time. We have also observed that incremental development in YOLO (You Only Look Once) make it more accurate and agile in object detection. We suggest that anomalies should be detected in real time and YOLO can play a vital role in anomalous activity.

Keywords: Autonomous self-driving, AI, deep learning, YOLO, R-CNN, Fast R-CNN, Faster R-CNN, SSD

1.1 Introduction

A crucial problem for the success of autonomous vehicles is ensuring safe driving. Before being released to the general public, self-driving cars must be thoroughly trained and tested. It should not compromise the safety of passengers or other traffic objects like vehicles, bikers, cyclists, pedestrians, etc. It should be thoroughly tested before the actual launch. Self-driving cars are controlled by software and the software must be trained in such a way that it can perform well under all circumstances or conditions. The following points need to be considered while developing software for autonomous vehicles.

Infrastructure: In the case of self-driving vehicles, infrastructure can be crucial. Almost the majority of the world’s roads and transportation infrastructure are now designed for human use. Autonomous vehicles will be required to operate inside existing infrastructure. For a self-driving vehicle, it is a challenging task to use current infrastructure. The software should be trained in such a way that it can easily adapt to the existing road infrastructure.

Traffic conditions: In real time, it is very difficult to predict traffic that what will happen next. It is almost impossible to accommodate all scenarios of traffic conditions while developing software for autonomous vehicles. However, AI-based algorithms should be developed in such a way that it can learn by itself with time and experience.

Weather condition: Weather can affect driving ability in autonomous vehicles. It may be possible that the inputs from various sensors and cameras get damaged due to bad weather, and in heavy rain or in a snowstorm various road streaks and lanes information can be hidden. An autonomous vehicle navigation system should be developed while considering weather conditions and it should be trained and tested in all weather conditions.

Software security: Self-driving cars completely depend on software, and software can be hacked and can be infected by viruses (a malicious computer code). Computer viruses can cause unexpected glitches in self-driving cars. These glitches can be harmful to self-driving cars especially while driving at a high speed. So the software needs to be secure for unauthorized access and viruses for safe driving.

1.1.1 Organization of Chapter

The rest of the chapter is outlined as follows. Section 1.2 gives the literature review. Section 1.3 describes an artificial intelligence approach in autonomous vehicles, section 1.4 outlines technologies inside an autonomous vehicle, section 1.5 shows major tasks in autonomous vehicle using AI, section 1.6 shows the benefits of autonomous vehicle, section 1.7 describes applications of autonomous vehicles. In section 1.8, anomalous activities and their categorization are described, while section 1.9 describes deep learning methods in an autonomous vehicle. Section 1.10 shows the working of YOLO, and section 1.11 shows the proposed method. Section 1.12 shows the proposed algorithm, while section 1.13 is a comparative study and discussion, and section 1.14 presents the conclusion of this chapter.

1.2 Literature Review

A security model has been suggested that can deal with three types of cyber-attacks for Electronic Control Units (ECUs). Over the years, the automobile sector has improved technology and there has been advancement in car production. To make the vehicle more comfortable and automated, companies are doing research on new technology. One of the advancements is that companies are replacing some mechanical parts with electronic components to introduce automation into vehicles. ECU is an electronic control unit that can communicate with other ECUs by messages. ECUs are modern technology that relies upon Control Area Network (CAN) and ensures that all the critical parts of a vehicle, like braking, engine, airbag, steering wheel, fuel indication, and acceleration are working properly. Due to the lack of security on the CAN bus network, it can be hacked and attackers can perform malicious activities in the ECU. The author’s security mechanism can solve three types of message attacks like fuzzy, Denial of service (DoS), and impersonation attacks. Deep learning-based network, Deep Denoising Autoencoder was adopted in the proposed security framework. Ecogeography-based optimization (EBO) algorithm was integrated with deep denoising autoencoder. For experimental data, malicious messages injected in CAN traffic were used. The experiment result showed that the proposed deep denoising autoencoder method outperforms the other machine learning models on three different CAN traffic datasets by achieving the highest hit rate and lowest miss rate [1].

A network called mIoUNe for detecting failure cases in semantic segmentation was proposed. To identify identical pixels and then label identical pixels with corresponding class is image segmentation. It may be possible that in the predicted semantic segmentation map, some pixels are labeled with the wrong class. For real-time applications like autonomous vehicles, this type of anomaly can lead to unsafe driving and results in accidents. The authors proposed a method using a neural network. Their network predicts the mean of the intersection of union (mIoU) to ensure that all the pixels were accurately classified. CNN and FCN were used in mIoUNet. Experimental results revealed that the proposed method achieved an accuracy of 93.21% mIoU prediction and 84.8% failure detection. In another experiment with HMG’s SVM camera acquisition dataset, the method achieved 90.51% mIoU prediction accuracy and 83.33% failure detection accuracy [2].

Road infrastructure will play a key role in the success of self-driving cars. There are many causes of traffic accidents and one of them is bad conditions. Android application was developed using OpenCV library to detect potholes and cracks in roads in real time. The proposed Automatic Pavement Distress Recognition (APDR) system was developed by combining the Android framework with the Open CV library. The system can detect road anomalies like fatigue cracks, longitudinal, potholes and transversal cracks. The Local Binary Pattern (LBP) feature cascade classifier was employed to train the system for positive samples and negative samples. A custom image dataset of the streets of Rome (Italy) was constructed for the experimental work. Using the LBP feature cascade classifier, the proposed Android system can detect road anomalies directly from live video frames. The system was tested on three android devices. Results showed that the system performed well in an older version android device as well as in a new device. This showed the portability of the proposed system [3].

Connected and automated vehicles (CAVs) capture the surrounding information from various sensors and cameras. Accurate information is very important for self-driving cars because autonomous vehicles are controlled by software. It may be possible that a sensor can provide an anomalous reading due to a faulty sensor or cyber-attacks. A faulty reading in an autonomous vehicle can leads to accidents. Therefore, real-time detection of anomalies is important. The experimental result of an anomaly detection method using CNN and Kalman filtering showed that the proposed approach can detect anomalies and identify their sources with high accuracy, sensitivity, and F1 score [4].

In autonomous vehicles, LiDAR, RADAR, cameras, and various...