Disney Research is redefining the intersection of AI and robotics, creating autonomous characters that seamlessly blend motion, perception, and expression to deliver truly immersive experiences. Mark Venables explores how deep reinforcement learning and real-time simulation are bringing these robotic performers to life, transforming entertainment and setting new benchmarks for AI-driven interaction.
The essence of Disney has always been storytelling. From its earliest animations to cutting-edge robotics, the company has continually redefined how audiences interact with characters. The development of autonomous robotic characters marks the next frontier, blending AI, reinforcement learning, and expressive motion to create lifelike interactions that immerse guests in new ways.
The vision is to create characters that feel alive, indistinguishable from their on-screen counterparts. “Our robotic character platform is the foundation of the BDX Druids, and over the past year, we have continued to refine and expand on our work,” Moritz Baecher, Associate Lab Director at Disney Research, says during his presentation at NVIDIA GTC in San Jose. “Our mission is to create autonomous robotic characters for our parks that can roam freely, interact with guests, and deliver a completely immersive experience. The goal is not to create mechanical systems but to develop characters that feel alive, indistinguishable from their on-screen counterparts.”
Engineering movement and personality
Robotic performance begins with movement. Achieving natural motion requires a combination of modular hardware and intelligent software, enabling characters to move, react, and emote seamlessly. The BDX Droid, a showcase of this innovation, features 14 actuators, five in each leg, allowing it to move expressively and convincingly. Integrating speakers, animated eyes, actuated antennas, and onboard Jetson processors provides additional layers of interaction. The droids are designed for sustained performances within theme parks with a removable battery offering nearly two hours of operation.
“Our approach combines modular hardware and intelligent software to design autonomous robotic characters rapidly,” Baecher adds. “The BDX Droid is a prime example of this. It features 14 actuators, five in each leg, enabling highly expressive motion. These actuators are meticulously characterised to ensure smooth and believable performances. The robot also incorporates essential show functions, including speakers for procedural audio, animated eyes on display screens, and actuated antennas for additional expressiveness. Two onboard Jetson processors power these capabilities, which provide the necessary computational power. The removable battery offers approximately one hour and 50 minutes of operation before needing a recharge.”
Beyond hardware, reinforcement learning plays a critical role in refining movement. Initial procedural animation from artists serves as a baseline for training AI models, refining and generalising movement across different environments. “One of the core challenges in robotic character development is ensuring that movement and behaviour feel natural,” Baecher continues. “To achieve this, we rely on deep reinforcement learning. The process begins with artists providing procedural animation and defining the target movement styles for the robot. These kinematic motions serve as input for reinforcement learning algorithms, which train control policies to generate smooth, stylised movements.”
This approach accelerates development while ensuring consistency across characters. By simultaneously simulating thousands of virtual instances, reinforcement learning condenses years of real-world training into mere days. Randomisation techniques expose models to weight distribution and terrain variations, preparing robots for real-world unpredictability.
Imbuing robots with emotion
Once movement is refined, the next challenge is expression. Emotionally engaging characters require subtlety. Small gestures, reactive postures, and gaze tracking all contribute to a sense of presence. AI-driven behavioural models combine these elements, allowing robots to generate real-time performances that align with character intent.
“Once a robot can walk in a natural and controlled manner, the next challenge is imbuing it with personality,” Baecher explains. “Expressive behaviour is critical to making these characters feel believable. To achieve this, we train policies based on short animation clips representing emotions such as happiness or shyness. For example, an artist-created animation of a shy droid turning away from a guest can be converted into a policy allowing the robot to generate similar movements autonomously. A separate model can enable an aggressive droid to maintain eye contact and track guests assertively. These models allow the robots to generate emotionally expressive performances without requiring manual input.”
Beyond physical expression, meaningful interaction is central to guest engagement. By capturing real-world data from creative operators who direct robotic performances, neural networks learn to generate character-driven interactions dynamically. This removes the need for manual control, allowing for more fluid guest experiences.
Advancing autonomy through perception
Autonomous movement is about more than pre-programmed routines. Genuine autonomy requires real-time perception and adaptive responses to external stimuli. Balancing, navigating complex spaces, and reacting to guests in an unscripted manner are all crucial for a truly immersive experience.
“To make these robotic characters more autonomous, we focus on two primary areas: balance and interaction,” Baecher says. “Balancing is critical for free-roaming characters, as they must be able to recover from external disturbances. Through simulation, we expose the robots to high-force pushes, enabling them to learn effective recovery strategies. This ensures they can remain stable, even when navigating uneven terrain or encountering obstacles.”
Perception-based interaction allows robots to move beyond scripted sequences. They build dynamic maps of their environment using onboard IMUs, depth sensors, and microphones and respond to changing conditions in real time. This integration enables tasks such as climbing stairs or dynamically adjusting gait in response to obstacles, an essential capability in theme park environments.
The future of motion generation
AI-driven motion generation is evolving. Instead of training individual behaviours separately, Disney is developing a generalised motion system capable of interpreting high-level commands, such as text descriptions or artistic direction, to produce real-time, physically viable animations.
“Currently, when we need to add a new behaviour, we must train a separate policy, which takes time,” Baecher explains. “To streamline this process, we are developing a system that uses a behaviour-agnostic tracking policy. By training on a dataset of human motion capture, the system can generalise movements across different scenarios without requiring retraining. This approach significantly improves efficiency and expands the range of possible robotic performances.”
Another critical advancement is the development of a physics-aware loss function. This function helps ensure generated motions remain physically viable, reducing the likelihood of failures. “By incorporating this into our motion diffusion models, we can create animations that are not only artistically compelling but also executable on real robots,” Baecher says.
Beyond the parks, these advancements influence how characters are integrated across Disney’s broader entertainment ecosystem. The BDX Droids, initially designed for theme parks, have since appeared in cruise lines and are now being adapted for film. “This reverse pipeline that has emerged is particularly exciting,” Baecher adds. “Traditionally, Disney brings characters from films into the parks. However, with the BDX Droids, we have taken an approach where the characters were first developed for the parks and are now being introduced into film. This underscores the growing role of robotics in shaping Disney’s storytelling across multiple platforms.”
Redefining robotics for entertainment
The future of robotic characters extends beyond entertainment into broader humanoid robotics research. The collaboration with Nvidia and Google DeepMind on the Newton simulator aims to refine actuator modelling and motion generation, bridging the gap between the physical and digital worlds.
“We are exploring new frontiers in humanoid robotics,” Baecher says. “Our research focuses on generating highly expressive, physically feasible motion. A key part of this work involves developing a new simulator, Newton, in collaboration with Nvidia and Google DeepMind. This simulator will integrate our animatronics and actuator modelling expertise, creating a unified platform for robotic character development.”
Unlike industrial robotics, which prioritises efficiency, Disney focuses on expression, storytelling, and characterisation. This requires innovations that balance mechanical efficiency with natural motion, pushing the boundaries of what is possible in AI-driven robotics.
“While the rest of the robotics industry focuses on functional performance in uncertain environments, our goal is to create believable, interactive characters in controlled settings,” Baecher concludes. “The challenge lies in custom-building each character while ensuring they can rapidly be brought to life with minimal programming.”
By reimagining the role of AI in robotics, Disney is enhancing guest experiences and shaping the future of expressive, autonomous characters in entertainment and beyond.
