Deep Learning in Drone Video Sensing

Deep learning (DL) has emerged as a transformative technology in remote sensing, revolutionizing the way remote-sensing images are analyzed and interpreted. This document provides a comprehensive review and meta-analysis of DL applications in remote sensing, covering various subfields, challenges, and future directions.

First, we review the evolution of remote-sensing image analysis, from traditional methods like support vector machines (SVM) and random forests (RF) to the resurgence of neural networks with the advent of DL. Since 2014, DL has gained prominence due to its superior performance in tasks such as land use and land cover (LULC) classification, scene classification, and object detection.

Next, the foundational DL models used in remote sensing deserves mention. Convolutional neural networks (CNNs) are the most widely used due to their ability to process multiband remote-sensing data. Recurrent neural networks (RNNs) are employed for sequential data analysis, while autoencoders (AEs) and deep belief networks (DBNs) are used for feature extraction and dimensionality reduction. Generative adversarial networks (GANs) have also gained traction for their ability to generate realistic data, making them useful for tasks like data augmentation.

Let's now review the state of DL in remote sensing with attributes such as study targets, DL models used, and accuracy levels. The usual focus is on LULC classification, object detection, and scene classification, with CNNs being the most frequently used model. High-resolution images (<10m), and urban areas were the most commonly analyzed. The median accuracy for scene classification was the highest (~95%), followed by object detection (~92%) and LULC classification (~91%).

Next, we delve into specific applications of DL in remote sensing. Image fusion, a fundamental task, benefits from DL's ability to characterize complex relationships between input and target images. Techniques like pan-sharpening and hyperspectral-multispectral fusion have been enhanced using CNNs and AEs. Image registration, essential for aligning images from different sensors or times, has seen advancements through Siamese networks and GANs. Scene classification and object detection, though similar, are distinguished by their focus on categorizing entire images versus identifying specific objects within images. DL has been instrumental in improving accuracy in both areas, though challenges like limited training data and object rotation variations persist.

LULC classification, a critical application, has seen significant improvements with DL. CNNs dominate this field, but other models like DBNs and GANs have also been explored. Challenges include the high cost of acquiring labeled training data and the need for methods that can handle medium- and low-resolution images. Semantic segmentation, which assigns labels to each pixel in an image, has benefited from fully convolutional networks (FCNs) but still faces challenges in balancing global context and local detail.

Object-based image analysis (OBIA) integrates DL with segmentation techniques to classify objects in remote-sensing images. While effective, the choice of parameters like patch size significantly impacts accuracy. Other emerging applications include time-series analysis, where RNNs are used to analyze sequential data, and the use of DL for tasks like accuracy assessment and data prediction.

Finally, we bring up the need for benchmark datasets, especially for medium- and low-resolution images, to standardize comparisons between DL algorithms. While DL has shown superior performance in many areas, challenges like training data limitations, network optimization, and real-world applicability remain. Future research should focus on addressing these challenges and expanding DL applications to underexplored areas like time-series analysis and image preprocessing.

Deep Learning has demonstrated immense potential in remote sensing, offering innovative solutions to longstanding challenges. This must be followed by the development of benchmark datasets, and the exploration of novel applications.