Researchers at the USC Viterbi School of Engineering’s Department of Computer Science are developing AI drones able to “learn” from challenges in the field.

Artem Molchanov, Tao Chen, Wolfgang Honig, James A. Preiss, Nora Ayanian and Gaurav Sukhatme authored a paper addressing how to train quadcopters to be more resilient and robust against disruptions.

“Currently, the controllers designed to stabilize quadcopters require careful tuning and even then, they are limited in terms of robustness to disruption and are model-specific,” Molchanov said in a USC news release.

“We’re trying to eliminate this problem and present an approach that leverages recent advancement in reinforcement learning so we can completely eliminate hand-tuning controllers and make drones super robust to disruptions.”

If the drone chooses the pre-determined correct operations or path, it receives a kind of “good-doggy cookie” — a mathematical reinforcement signal. From such positive reinforcement, a drone can infer which actions are most desirable.

Researchers explain:

“Over the course of 24 hours, the system processes 250 hours of real-world training. Like training wheels, learning in simulation mode allows the drone to learn on its own in a safe environment, before being released into the wild. Eventually, it finds solutions to every challenge put in its path.”

“Controlling a drone requires a lot of precision. Especially when something sudden occurs, you need a fast and precise sequence of control inputs,” Molchanov said.

“Reinforcement learning is inspired by biology—it’s very similar to how you might train a dog with a reward when it completes a command. We keep slightly changing the simulator, which allows the drone to learn to adapt to all possible imperfections of the environment.”

Next, the team re-located the trained controller to real drones developed in Ayanian’s Automatic Coordination of Teams Lab. Within an indoor drone facility, they threw challenges at the drones by going all Jean-Claude Van Damme on them.

“The drones were successful in correcting themselves from moderate hits (including pushes, light kicks and colliding with an object) 90 percent of the time. Once trained on one machine, the controller was able to quickly generalize to quadcopters with different dimensions, weights and sizes.”