Through NSF grant, researchers creating advanced autopilot to mimic the ‘Sully Factor’

It’s called the “Miracle on the Hudson.” On January 15, 2009, U.S. Airways Flight 1549 left LaGuardia Airport in New York City en route to Seattle via Charlotte. Shortly after takeoff, the plane hit birds and lost two engines. Faced with few options, the captain, Chesley “Sully” Sullenberger, took elusive action and landed the plane in the Hudson River, saving the lives of the 155 people on board.

While Sullenberger’s experience – a fighter pilot and training officer with the United States Air Force and 30 years as a commercial airline pilot – no doubt was a factor in his quick decision making, not all passengers are lucky enough to have that experience guiding their flights.

Through at National Science Foundation grant, a team of researchers at the University of Illinois at Urbana-Champaign’s Grainger College of Engineering and the Georgia Institute of Technology are working on an advanced autopilot, which will autonomously evaluate unforeseen circumstances, take the best course of action, and land the plane safely.

“We cannot be all Captain Sully,” said Lui Sha, the Donald B. Gillies Chair in Computer Science at Illinois, an expert in safety critical real-time systems, and the principal investigator on this project. “He is a genius. He is to pilots what Michelangelo is to painters. However, not all pilots have his expertise.”

Sha has been a member of the Office of the Secretary of Defense’s Avionics Advisory Team and has served on the NASA Advisory Council. He is among those credited with saving the Mars Pathfinder when it failed in 1997. Sha is joined on this all-star team by Naira Hovakiyman, the world’s top specialist in robust flight and L1 adaptive control and a professor mechanical science and engineering at Illinois; Evangelos Theodorou, who leads the Autonomous Control and Decision Systems Laboratory at Georgia Tech and is an expert in stochastic control and machine learning; and Petros Voulgaris, a professor of aerospace engineering at Illinois who also specializes in robust networked control systems.

“Even as smart as Sully is, it still took him 32 seconds to understand it was a bird strike, that the engines were gone, and decide to land in the river,” Hovakimyan noted. “We want to build a flight control system that could handle a lot of uncertainties instantly. To make this happen, we need high fault-tolerant computing infrastructure that can withstand high-stress situations (CPU, GPU, parallelization and so on) and the full detection isolation to immediately work when something happens. All of this is a level of autonomy that is missing from the current airplane, but we can build it into the autopilot based on the science we have.”

Hovakimyan has been working on the L1 Adaptive Control theory, which can aid a pilot to regain control of an airplane in sudden and drastic circumstances, for about 15 years. Her research was validated by NASA on a subscale commercial jet and by the United States Air Force on a Learjet and an F16.

The new autopilot will make it less dependent on the pilot and act immediately. It will combine image processing from cameras, Google maps, etc., as well as on-board computers, and include a built-in sensor redundancy.

“This is a case of an almost worst-case scenario that can yet be remedied by human intelligence,” Voulgaris said. “It is this capability to defend against these potentially catastrophic failures that we want to build into autonomy without being overly conservative.

Sha’s insights include how to keep an airplane within its flight envelope by calculating its acceleration, rotation, angle, and position.

“It turns out that the stability envelope on an airplane is designed conservatively,” Sha said. “In passenger jets, they live within a large safety margin. The autopilots are designed to operate in a normal operational window. When it goes outside of it, the pilot has to take action. Our system will be the new generation autopilot that would have more autonomous capabilities than the existing ones so we can be safer even with less experienced captains.”

The team will take advantage of Theodorou’s expertise in artificial intelligence in building the new system. Last month, a pilot on a Russian flight made a similar maneuver, landing a plane in a corn field after a bird strike. In response, the Russians are creating courses on how what to do in these unforeseen circumstances.

“When I look into the sequence of events and how Captain Sullenberger’s reacted I am thinking, ‘Here is an example of predictive decision making,'” Theodorou said. “Humans and many biological organisms have the capability to perform reactive as well as more elaborate predictive decisions. To me this is indication of decision-making processes that take place in different time scales. Our goal is to develop algorithms that have similar capabilities and can satisfy safety-critical constraints and requirements.”

“What they fail to understand is that the human brain has all these neurons that gets trained over our lifetime,” Sha said. “We do some things automatically when we are driving, for instance. We want to build a fault-tolerant computing infrastructure that can withstand high-stress situations with enough redundancies to mimic the human brain of an experienced person.”

The team is striving to build an intelligence control system, which will create scenarios never before encountered and for the system to, among other things, determine whether the flight objective still makes sense. If the scenario is too far out of the flight objective, it will be able to reexamine the objective and complete a different mission, which might be to make an emergency landing somewhere other than an airport.

Sha compares the new system to that of a driving instructor than can take over if something goes terribly wrong.

“We want to integrate the advanced robust adaptive control, this syntax architecture with AI,” he said. “It watches the behavior of a vehicle and the control of a complex software. If a behavior isn’t stable, it can take over in real time without missing a beat.”

The research team is already receiving some interest from industry. For instance, The Thales Group, a French multinational company that designs and builds electrical systems and provides services for the aerospace, defense, transportation and security markets, will visit the team on the Illinois campus in October.

“We are not saying we are building the magic that would do what is impossible,” Hovakimyan concluded. “We are simply reproducing the Sully capabilities. He was able to see what was around, understand what was at fault, and quickly make a decision on the best action based on all of this information using his past experience.”

###