News Research

Depth Transfer: Learning to See Like a Simulator for Real-World Drone Navigation

By hyyuPosted on
Time to Read:-words

🧠 Bridging the Gap Between Simulation and Reality

Training drones to navigate complex environments autonomously is one of the central challenges in robotics. While simulators like IsaacGym make reinforcement learning (RL) efficient and safe, transferring these policies to the real world remains difficult — primarily due to the visual gap between simulated and real-world depth perception.

Our research, “Depth Transfer: Learning to See Like a Simulator for Real-World Drone Navigation”, published in IEEE Robotics and Automation Letters (RA-L, October 2025), introduces a novel approach to close this gap through depth domain adaptation.

🚀 The Core Idea: Depth Transfer via Domain Adaptation

We propose a depth transfer framework that allows an RL policy trained purely in simulation to perform robustly in the real world — without retraining.

At the heart of this approach lies a Variational Autoencoder (VAE) that learns compact latent representations of ideal depth images from simulation. During deployment, the encoder is refined via domain adaptation using stereo depth data from real environments.
A Gradient Reversal Layer (GRL) aligns the latent features from simulation and reality, ensuring the drone “sees” the world through the same internal representation.

This enables zero-shot transfer — the policy trained on simulated ground-truth depth can directly navigate in real forests, cluttered indoor spaces, and even in photo-realistic simulators like AvoidBench.

🧩 Key Innovations

  • Feature-level Domain Adaptation: Aligns depth features from simulation and reality without paired training data.

  • ResNet-based VAE + Min-Pooling Dilation: Preserves fine obstacle details, improving reconstruction accuracy and navigation safety.

  • Temporal Representation via LSTM: Captures motion consistency in dynamic navigation scenes.

  • Plug-and-Play Deployment: The policy trained in IsaacGym transfers seamlessly to real drones.

🌲 Real-World and Simulation Results

  • In IsaacGym, domain adaptation doubles the success rate when switching from simulated to stereo depth input.

  • In AvoidBench, the method outperforms MAVRL, Ego-Planner, and Agile-Autonomy — despite being trained entirely in IsaacGym.

  • In real-world forest tests, performance improved significantly, with faster and smoother trajectories.

📦 Resources