By Celia Pizzuto
The advancement of autonomous systems has reached new heights with vision-based drones, a cutting-edge technology set to revolutionize various industries. At the 2024 AI for Good Global Summit, Davide Scaramuzza, a professor of Robotics and Perception at the University of Zurich, presented groundbreaking research demonstrating how these drones can operate independently using only onboard cameras, without the need for GPS. His presentation emphasized the critical role of perception in achieving full autonomy and unlocking new capabilities for these flying robots.
The Power of Perception
Perception plays a critical role in robotics. According to Scaramuzza, without it, no robot can be fully autonomous or truly useful. He illustrated this point with examples, including significant improvements in consumer robotics like vacuum cleaners and the Boston Dynamics Atlas robot.
“Perception, also called computer vision, is making mobile robots autonomous,” he stated.
For instance, the Roomba vacuum cleaner’s efficiency increased dramatically after integrating a camera, enabling it to navigate more intelligently. Similarly, Atlas can perform complex tasks thanks to its perception capabilities, which allow it to map its environment and make decisions about where to move.
He also highlighted NASA’s Mars Ingenuity helicopter, the first autonomous robot on another planet, which relied entirely on onboard cameras and computers for its numerous flights. Scaramuzza underscored that while robots are becoming more autonomous, they still fall short of human performance in agility, versatility, and robustness. He noted that humans are still preferred in most applications, highlighting the gap between current robotic capabilities and human skills.
“Humans are truly robust, and machines are not yet a human level,” he stated.
The Future of Autonomous Flight
In his lab, Scaramuzza and his team have been developing drones with advanced autopilots that aim to outperform human pilots. These drones are particularly challenging to navigate because they operate in three dimensions, unlike ground-based robots. He showed a video demonstrating the high skill level required for human-operated drones and explained that current commercial drones do not fly at optimal speeds due to challenges in perception and control, such as motion blur and aerodynamic effects.
Scaramuzza emphasized the importance of increasing the agility of drones to enhance their productivity, pointing out that drones can only operate for 20-30 minutes per flight. This limited flight time restricts the range and number of tasks they can accomplish. By making drones more agile and faster, they can cover greater distances within the same amount of time, thereby becoming more efficient and useful for various applications, ranging from search and rescue to space exploration.
To push the limits of drone agility, Scaramuzza’s team collaborated with drone racing pilots, developing a small, highly agile drone capable of accelerating from 0 to 100 km/h in less than a second. “We built a truly agile drone,” he explained, detailing how the drone uses machine learning and onboard cameras to perform complex maneuvers autonomously, without GPS or human intervention. Scaramuzza noted that while these achievements are impressive, the ultimate goal is to extend these capabilities to a broader range of robots and applications.
The Role of Simulation and AI
A key challenge in developing autonomous drones is gathering the vast amounts of data needed to train perception and control algorithms.
“But when you start using AI, you need a lot of data. Where do you get all the data to train this control and perception algorithms?” Scaramuzza asked?
He explained that real-world data collection is impractical due to legal restrictions and the time required. Instead, his team uses simulation to generate large datasets quickly and safely. “In simulation, you can simulate an unlimited amount of data. In only one night, we can simulate 100 drones flying hundreds of forests” he remarked, highlighting the efficiency of this method.
“Thanks to AI combined with the proper perception algorithms, we can fly drones in a way that seem very similar to the way humans are flying drones,” he explained.
This approach has enabled the lab to achieve significant advancements, such as flying drones through forests at high speeds, outperforming commercial models that struggle with vision at high velocity. Scaramuzza emphasized that while simulations are invaluable, real-world robustness and versatility remain significant challenges for autonomous systems.
Achievements and Real-World Applications
The highlight of Scaramuzza’s presentation was a demonstration showing the results of a race between their AI-powered drone, Swift, and top human drone racing pilots, including world champions. Swift won the majority of the races, demonstrating that AI drones could potentially surpass human pilots in specific controlled environments. However, Scaramuzza cautioned that these results were achieved under controlled conditions, and AI drones are not yet capable of matching human versatility and decision-making in unpredictable real-world scenarios. “We have to be cautious as researchers,” he advised, acknowledging the limitations of current technology.
The research conducted in Scaramuzza’s lab has led to several successful spin-offs, including:
- Suind: A company focused on creating agile drones for applications such as crop monitoring and wildfire prevention, utilizing onboard perception and AI.
- Fotokite: A tethered drone system for first responders, providing situational awareness during emergencies.
- Zurich Eye: Acquired by Facebook, this company worked on developing the Meta Quest VR headset, using algorithms initially designed for drone navigation to enhance virtual reality experiences.
The Road Ahead for Autonomous Drone Technology
In response to audience questions, Scaramuzza discussed the limitations of current onboard computation and the potential of cloud-based processing to overcome these challenges. He noted that with advancements in larger language and visual models, the computational power required is increasing, making offboard computation a viable alternative as long as latency remains manageable. “We are moving away from onboard computation to offboard computation, as long as it’s available,” he explained.
The presentation at the AI for Good Global Summit showcased the cutting-edge advancements in vision-based drones, highlighting the potential these technologies hold for future innovations in robotics and automation. There is a clear need for further exploration and collaboration in the field of autonomous drones, particularly to enhance robustness and versatility. As these technologies continue to advance, they are expected to expand their applications across diverse areas, including search and rescue, environmental monitoring, and beyond.
Watch the full session:
