Drones and AI used to enhance lost person search and rescue

With warmer weather right around the corner, many outdoor enthusiasts are preparing to hit the trails in regional, state, and national parks.

All across the country, search and rescue organizations are preparing, too.

In 2017 alone, the United States National Park Service deployed almost 3,500 search and rescue missions in national parks. And while search and rescue personnel tend to be highly effective and skillfully trained, they still face a daunting task dominated by a ticking clock – a lost person’s chances for survival drop dramatically after the first 18 hours.

Thanks to a $1.5 million grant from the National Science Foundation, a group of Virginia Tech engineers hopes to redefine these search and rescue protocols by teaming up human searchers with unmanned aerial robots, or drones.

In efforts led by Ryan Williams, an assistant professor in the Bradley Department of Electrical and Computer Engineering within the College of Engineering, these drones will use autonomous algorithms and machine learning to complement search and rescue efforts from the air. The drones will also suggest tasks and send updated information to human searchers on the ground.

Using mathematical models based on historical data that reflect what lost people actually do combined with typical searcher behavior, the researchers hope this novel approach of balancing autonomy with human collaboration can make searches more effective. The team has received support from the Virginia Department of Emergency Management and will work closely with the local Black Diamond Search and Rescue Council throughout the project.

“Human searchers are excellent at what they do,” said Williams. “Drones are unlikely to replace people on the ground because searchers are too good at their jobs. However, what drones can do is address these niche problems of the search process by providing large-scale data that can help a search team make better decisions.”

For example, drones may explore treacherous terrain that’s difficult for human searchers to reach, or they could save time by gathering information about areas of the search environment that are relatively unknown.

Nicole Abaid, an assistant professor in the Department of Biomedical Engineering and Mechanics, is using historical data gathered from more than 50,000 documented lost person scenarios to build the mathematical model that will help the drones decide where to go and how to search. The data comes from the work of famed search and rescue expert Robert Koester, who will consult on the project.

“From the historical search data, we know that certain types of people tend to do certain things when they’re lost,” said Abaid. “For example, people with cell phones tend to move up in elevation as they try to get service, while an elderly person might not travel very far.”

Abaid plans to build over 30 lost person profiles into the model that incorporate information like age, mental status, and activity (hiking, horseback riding, hunting, etc.). She’ll also use topographical data from ArcGIS maps that can shed light on how people typically move through terrain.

“The overall goal is to create an autonomous, scalable system that can make search and rescue processes here and elsewhere more effective simply by intelligently incorporating existing technology,” said Abaid.

Williams, whose family has been involved with the Black Diamond Search and Rescue Council since its founding over 30 years ago, stressed the importance of making the system as unobtrusive as possible to searchers on the ground.

“Most of the data a drone will pick up from sensors, whether it be from cameras, thermal imaging, or Lidar surveys, will be really uninteresting,” said Williams. “We don’t want to interrupt human searchers to show them video feed of a piece of trash on the ground, for instance. That wastes precious time.”

To address that problem of when and how the drones will interrupt searchers to provide new information, the team includes Nathan Lau, an assistant professor in the Grado Department of Industrial and Systems Engineering who specializes in human-computer interaction.

James McClure, a computational scientist in Advanced Research Computing, rounds out the team and will provide on-the-ground processing and communications support by designing a wearable, backpack-based server.

The researchers agree that improving an already effective system implemented by human teams will be challenging. However, they hope their close collaboration with human searchers throughout the process will produce meaningful results for real-world search and rescue practitioners.

“Our group is very driven,” said Amanda Hashimoto, an engineering mechanics doctoral student who assists with modeling aspects of the project. “The idea of being able to help save people’s lives is pretty motivating.”

###