Aerial imaging technique improves ability to detect and track moving targets through thick foliage

In forests where the foliage is thick, it can be challenging to detect and track moving targets, such as people and animals, using the current technology for collecting aerial images and videos. Researchers have developed a drone-operated 1D camera array that uses airborne optical sectioning to detect and track moving people in a dense forest. This new technique can be a helpful addition to the technology used in search and rescue missions. 

The team published their findings on April 22, 2022, in the Journal of Remote Sensing.

 

The team uses airborne optical sectioning, a wide synthetic aperture imaging technique that employs conventional drones to sample images while flying above the forest. In synthetic aperture imaging, researchers can remotely map environments using electromagnetic signals. The airborne optical sectioning technique provides real-time computational performance, high spatial resolution, and also is applicable to other wavelengths. Airborne optical sectioning was limited to applications wherein the target was static. Using it to detect moving targets has led to motion blur in the images.

 

The research team’s light-weight, drone-operated 1D camera array that supports parallel airborne optical sectioning sampling has overcome these challenges. In the past, 1D and 2D camera arrays have been used for various visual effects, but they have not been applied for aerial imaging, especially in use with drones, because of their size and weight. The team’s 1D camera is very light, weighing less than 1 kilogram.

 

The team showed that the color anomaly detector benefits significantly using the airborne optical sectioning integral images, compared to conventional raw images. Color anomaly detection is a technique used for aerial image analysis in search and rescue applications because it is not dependent on the environment temperature, as in thermal imaging. Color anomaly detection finds pixels or clusters of pixels in an image with significant color differences in comparison to their neighbors. However, color anomaly detection fails where there is dense foliage or vegetation. The team demonstrated that their technique can detect and track moving people through dense forest foliage. In their field experiments, the color anomaly detection improved to an average of 97 percent compared with 42 percent when using conventional raw images or video frames.

 

Mounted on a drone, their camera payload captures multiple aerial images while sampling the forest environment. Some of the images are occluded, with the view obstructed by foliage. Other images are unoccluded. The team integrates the occluded and unoccluded images resulting in a common target point on the ground. The image integration provides the benefits of color anomaly detection, giving the researchers a clearer picture of any people moving on the ground, but shielded from the view of the human eye because of the foliage. “While detecting and tracking moving targets through foliage is difficult, and often even impossible, in regular aerial images or videos, it becomes practically feasible with image integration—which is the core principle of airborne optical sectioning,” said Rakesh John Amala Arokia Nathan, a researcher at Johannes Kepler University Linz and the first author on the paper.

 

Besides being useful in detecting people, this technique can be helpful in tracking other targets such as vehicles or animals. “This might impact many application domains, such as search and rescue, surveillance, border control, and wildlife observation,” said Nathan.

 

At this time the color anomaly detection is limited to detectable target colors. Looking ahead to future research, the team might explore a combination of color, thermal, and time (or motion) channels for anomaly detection. “Furthermore, the implications of parallel-sequential sampling strategies and other sampling devices, such as re-configurable drone swarms instead of camera arrays with a fixed sampling pattern, have to be explored,” said Nathan.

 

The research team includes Rakesh John Amala Arokia Nathan, Indrajit Kurmi, and Oliver Bimber from the Institute of Computer Graphics, Johannes Kepler University Linz, Linz, Austria; and David C. Schedl, from the University of Applied Sciences Upper Austria, Hagenberg, Austria.

 

The research was funded by the Austrian Science Fund and by the State of Upper Austria and the Austrian Federal Ministry of Education, Science and Research.

The team uses airborne optical sectioning, a wide synthetic aperture imaging technique that employs conventional drones to sample images while flying above the forest. In synthetic aperture imaging, researchers can remotely map environments using electromagnetic signals. The airborne optical sectioning technique provides real-time computational performance, high spatial resolution, and also is applicable to other wavelengths. Airborne optical sectioning was limited to applications wherein the target was static. Using it to detect moving targets has led to motion blur in the images.

 

The research team’s light-weight, drone-operated 1D camera array that supports parallel airborne optical sectioning sampling has overcome these challenges. In the past, 1D and 2D camera arrays have been used for various visual effects, but they have not been applied for aerial imaging, especially in use with drones, because of their size and weight. The team’s 1D camera is very light, weighing less than 1 kilogram.

 

The team showed that the color anomaly detector benefits significantly using the airborne optical sectioning integral images, compared to conventional raw images. Color anomaly detection is a technique used for aerial image analysis in search and rescue applications because it is not dependent on the environment temperature, as in thermal imaging. Color anomaly detection finds pixels or clusters of pixels in an image with significant color differences in comparison to their neighbors. However, color anomaly detection fails where there is dense foliage or vegetation. The team demonstrated that their technique can detect and track moving people through dense forest foliage. In their field experiments, the color anomaly detection improved to an average of 97 percent compared with 42 percent when using conventional raw images or video frames.

 

Mounted on a drone, their camera payload captures multiple aerial images while sampling the forest environment. Some of the images are occluded, with the view obstructed by foliage. Other images are unoccluded. The team integrates the occluded and unoccluded images resulting in a common target point on the ground. The image integration provides the benefits of color anomaly detection, giving the researchers a clearer picture of any people moving on the ground, but shielded from the view of the human eye because of the foliage. “While detecting and tracking moving targets through foliage is difficult, and often even impossible, in regular aerial images or videos, it becomes practically feasible with image integration—which is the core principle of airborne optical sectioning,” said Rakesh John Amala Arokia Nathan, a researcher at Johannes Kepler University Linz and the first author on the paper.

 

Besides being useful in detecting people, this technique can be helpful in tracking other targets such as vehicles or animals. “This might impact many application domains, such as search and rescue, surveillance, border control, and wildlife observation,” said Nathan.

 

At this time the color anomaly detection is limited to detectable target colors. Looking ahead to future research, the team might explore a combination of color, thermal, and time (or motion) channels for anomaly detection. “Furthermore, the implications of parallel-sequential sampling strategies and other sampling devices, such as re-configurable drone swarms instead of camera arrays with a fixed sampling pattern, have to be explored,” said Nathan.

 

The research team includes Rakesh John Amala Arokia Nathan, Indrajit Kurmi, and Oliver Bimber from the Institute of Computer Graphics, Johannes Kepler University Linz, Linz, Austria; and David C. Schedl, from the University of Applied Sciences Upper Austria, Hagenberg, Austria.

 

The research was funded by the Austrian Science Fund and by the State of Upper Austria and the Austrian Federal Ministry of Education, Science and Research.

 

---

---