Google DeepMind has created a robotic table tennis player that can compete at an amateur human level, showcasing advancements in bridging the “Sim to Real” gap in robotics. The robot learns through a combination of simulation and real-world fine-tuning, allowing it to adapt to human gameplay and receive positive feedback from players.
Google DeepMind has developed a robotic table tennis player that achieves amateur human-level performance, marking a significant advancement in robotics research aimed at reaching human-like speed and adaptability in real-world tasks. This robot represents the first learned agent capable of competing at this level in table tennis, a sport that requires complex skills and strategic gameplay. A key focus of the research is overcoming the “Sim to Real” gap, which refers to the challenge of transferring skills learned in a simulated environment to real-world applications. Recent advancements from Google DeepMind and Nvidia have shown promising results in bridging this gap.
One of the primary challenges faced by the researchers was the lack of data on robots playing table tennis against human opponents. Unlike other sports, there is limited footage available for training purposes. To address this, the team utilized a physics simulator called Mujoco, which is open-source and available on GitHub. This simulator allowed the AI agent to learn basic skills, such as forehand and backhand serves, and to adapt to various styles of play, including different spins applied by human opponents.
The training process involved an iterative approach known as IS2, which bootstraps from a simple model of human behavior. Initially, the robot collected data on human interactions without actively participating, which helped create a model of human behavior. This model was then used to train the robot in simulation. Once the robot’s policy converged, it was transferred to the physical robot for fine-tuning, allowing it to improve its cooperative play with humans. This cycle of training and fine-tuning continued until the robot could effectively interact with human players.
The robot employs a spin classifier to determine the type of spin applied to the ball by human players, which informs its low-level controllers (LLCs) to execute appropriate responses. The high-level controller (HLC) makes strategic decisions about gameplay, such as shot placement and risk management. The simulation also accounted for the physical properties of the paddle, ensuring that the robot’s movements closely mirrored those of a human player. This attention to detail in the simulation process has contributed to the robot’s successful performance in real-world scenarios.
Feedback from human players who interacted with the robot indicated a positive experience, with most participants finding it fun and engaging to play against. The robot was rated as neither too easy nor too hard, and players expressed a willingness to play with it again. This development not only showcases the potential of AI in competitive sports but also highlights the broader implications of training robots in simulations and transferring those skills to real-world applications. Google DeepMind’s commitment to open-sourcing their work allows others to explore and implement similar robotic learning strategies, further advancing the field of robotics.