Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction

One of the crucial situations autonomous cars and driver-assist features need to navigate is pedestrian crossing. While it is easier for a system to process that there is a pedestrian already crossing the street, predicting whether a pedestrian will or will not cross is more difficult. To approach this issue, Stanford researchers have developed a framework based on graph convolution to uncover the spatiotemporal relationships in the scene for reasoning about pedestrian intent. The scene graph is built on top of the segmented instances within and across video frames for both pedestrian-centric and location-centric models. These are then fed into a gated recurrent unit which predicts future crossing actions. Early results from the spatiotemporal relationship model predict intent to cross accurately ~80% of the time about 1s before a pedestrian actually crosses.

Image description: Diagram of the evaluation process. Photo credit: IEEE Robotics and Automation Letters 5.2 (2020): 3485-3492.

Stage of Research
Prototype