Research paper presented at the 2018 International Conference on Robotics and Intelligent System (ICRIS 2018) in Amsterdam and won a best paper award. Published by ACM in the ICMSCE 2018 Proceedings of the 2018 2nd International Conference on Mechatronics Systems and Control Engineering.
Robotic following has many practical applications and involves several cognitive sub-tasks including object identification, localization, tracking, path planning, and motion control. We propose a system that uses a single deep neural network (DNN), with video camera pixels as the only input, to handle all the cognitive perception and visuomotor control functions needed to perform robotic following behaviors. Our approach combines 1) end-to-end deep learning for inferring motion control outputs from visual inputs, 2) multi-task learning for simultaneously producing multiple outputs with the same DNN, and 3) spatio-temporal deep learning for perceiving motion across multiple video frames. We use simulations of truck platooning to quantitatively show that spatio-temporal DNNs increase driving accuracy and improve machine perception of scene kinematics. Experiments conducted with mobile robots in a laboratory test track show real-time embedded systems performance and indicate that an end-to-end deep learning robot is able to follow its leader for long periods of time, while keeping within lanes and avoiding obstacles.