Robotics
Semantic mapping
Definition
Semantic mapping builds a spatial representation that associates places or geometry with meanings such as object classes, instances, room types, or attributes. It gives a robot information about what is in the environment as well as where surfaces are.
Updated
Attach meaning to spatial estimates
A geometric map can record occupied cells, surfaces, or landmarks without saying what they represent. A semantic map associates those locations with labels or entities, such as floor, door, table, tool, or a particular object instance. A robot can then ask where a usable surface is instead of searching only by coordinates.
Semantic mapping often combines pose estimation, geometric reconstruction, and repeated predictions from a perception model. The map provides a common spatial frame in which evidence from several views can be accumulated.
Fuse observations across views
SemanticFusion combines a dense RGB-D SLAM system with convolutional-network predictions. Long-term correspondences from the geometric reconstruction let the system probabilistically fuse semantic predictions from multiple viewpoints into a dense 3D map. The reported results concern that system and its datasets, rather than every semantic-mapping pipeline.
A semantic map differs from single-image segmentation because labels persist in a spatial representation across frames. It also differs from a 3D dynamic scene graph, which explicitly represents entities and relations at several levels; a semantic map can be a labelled surface or grid without that graph structure.
Labels can be confidently wrong
Pose drift, depth error, occlusion, and calibration error can place a correct label on the wrong surface. Correlated predictions from similar camera views do not provide independent evidence. A closed-set model may also force an unfamiliar object into one of its known classes.
Dynamic objects add another problem because fusing them into a static map leaves stale geometry and labels. A robot using semantic maps for manipulation or navigation still needs uncertainty handling, updates, and geometric safety checks rather than treating every label as ground truth.
Sources
Related terms
Simultaneous localization and mapping
Simultaneous localization and mapping is the joint estimation of a robot's state and a map of its environment from sensor observations. It is commonly abbreviated SLAM.
3D dynamic scene graph
A 3D dynamic scene graph is a layered graph that represents places, objects, people, and other spatial entities as nodes connected by geometric, semantic, and time-dependent relations. It gives a robot a structured scene representation above raw geometry alone.
Open-vocabulary object detection
Open-vocabulary object detection locates objects in an image and associates them with text-defined categories beyond a fixed closed list used during detector training. It combines spatial detection with vision-language representations that can be queried using class names or phrases.
Point cloud
A point cloud is a collection of points representing sampled locations in space, usually with three-dimensional coordinates. Individual points may also carry attributes such as color or return intensity.