Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision

Pedestrian movement prediction is a critical aspect of driver-assistance and autonomous cars. This requires predicting both human poses and human trajectories based on keypoints in an egocentric setting. One of the key challenges is the lack of annotated egocentric video datasets with dense pose annotation. This required the inventors to use pretrained models to generate noisy annotations which were then used to train the pedestrian forecasting models. The noisy annotations were disentangled to separate locomotion into global movement (pedestrian trajectory) and local movement (pose keypoint movements). This allows the system to predict human locomotion even when the pose keypoint data is incomplete and merges at the end to allow for faster prediction.

Stage of Research
Prototype