Robotics
Scene flow
Definition
Scene flow is a three-dimensional motion field that assigns a displacement or velocity to visible points in a dynamic scene over time. Robots can estimate it from stereo, RGB-D, or lidar observations to reason about moving geometry.
Also known as: 3D scene flow
Updated
Motion in three dimensions
Optical flow describes apparent two-dimensional image motion. Scene flow lifts the idea into three dimensions by estimating how observed surface points move through space. The original three-dimensional scene-flow work formulates the field from multiple camera views and relates it to image motion and changing scene structure.
An estimate can come from stereo images, depth cameras, successive point clouds, or fused sensors. It may represent displacement between two frames or velocity over a time interval. The coordinate frame and whether camera ego-motion has been removed must be stated before the vectors can be interpreted.
Dynamic perception for robots
Scene flow can help separate moving objects from the static environment, predict where occupied space is changing, or provide motion cues for tracking and navigation. FlowNet3D estimates flow directly between point clouds and reports applications to scan registration and motion segmentation, with experiments on synthetic data and KITTI lidar scans. Those results apply to that network, training data, and evaluation protocol.
Scene flow differs from point-cloud registration. Rigid registration seeks one transform that aligns a whole scan or object. Scene flow can assign different motions to different points, allowing articulated objects and several independently moving bodies.
Visibility and correspondence are difficult
Points can disappear through occlusion, leave the sensor range, or appear for the first time. Sparse lidar sampling may not observe the same physical surface point in consecutive scans. Reflective surfaces, depth noise, timing offsets, and motion distortion create additional ambiguity.
The estimate also mixes object motion with sensor motion unless ego-motion is modeled. Smoothness assumptions can blur motion boundaries, while a learned estimator can inherit object, speed, weather, and sensor biases from its training set. Scene flow is therefore an uncertain perception output, not a guarantee of future free space or collision avoidance.
Sources
Related terms
Point cloud
A point cloud is a collection of points representing sampled locations in space, usually with three-dimensional coordinates. Individual points may also carry attributes such as color or return intensity.
Lidar
Lidar measures distance using emitted laser light and its return from surfaces. Repeated range measurements across directions can form a spatial scan or three-dimensional point cloud.
Point cloud registration
Point cloud registration estimates a spatial transformation that aligns two or more point clouds in a common coordinate frame. Robots use it to relate scans for mapping, localization, inspection, and object pose estimation.
Visual-inertial odometry
Visual-inertial odometry estimates a moving system's motion by combining camera observations with inertial measurements. It typically estimates position, orientation, velocity, and sensor biases over time.