Scene Representation and Mapping

Perception answers where an object is right now. This module answers the other question a robot needs settled before it can move: what does it remember about the space it is standing in, and what can it ask of that memory.

The answers form a ladder. Each rung stores something different, and the choice is not a matter of fidelity, because a rung that draws the scene beautifully may be unable to answer the one question a planner needs.

The ladder, and what each rung can answer

Three questions separate the rungs. Can the map say whether a given cell is free, so a planner can route through it. Can it supply a surface normal, so a contact solver knows which way the surface faces. Can it render the scene from a viewpoint the sensor never occupied. No representation on this ladder answers all three well, and the disagreements are structural rather than a matter of implementation quality.

Switch the representation selector below from the occupancy grid to the Gaussian splat and watch what happens to the region behind the occluder, which no sensor ray ever reached. Then move the resolution slider and notice what does not change.

The occupancy grid holds that region as an explicit unknown. The splat fills it with plausible geometry, rendered at the same confidence as the surfaces that were actually measured. Nothing in the splat's store distinguishes the two, and that single difference is the reason the rest of this module is worth reading.

Point clouds and occupancy grids

A point cloudpoint cloudA set of 3D points, usually with no ordering and no connectivity, which is what a depth camera or a lidar produces once its measurements are back-projected through the camera intrinsics. The awkwardness for learning is that the set is unordered, so a network reading it must be invariant to permutation of its own input. PointNet answered that with a shared per-point encoder followed by a symmetric pooling function. PointNet++ groups metric-space neighbourhoods and applies local PointNets recursively to learn features at increasing scales. These are summaries, not a guarantee that all geometry survives pooling; density-adaptive grouping addresses sparsely sampled neighbourhoods. is an unordered set of 3D points. PointNet treats its input this way and requires invariance to the order in which those points are supplied Qi 2017↓PointNet: Deep Learning on Point Sets for 3D Classification and SegmentationCharles R. Qi, Hao Su, Kaichun Mo et al., CVPR 2017. Its weakness is silence. A region with no points might be empty or might be unobserved, and the store cannot tell you which.

Moravec and Elfes projected wide-angle sonar measurements onto a two-dimensional horizontal occupancy gridoccupancy gridA map that divides space into cells and stores, per cell, the probability that it is occupied. Moravec and Elfes introduced it for wide-angle sonar: each range reading constrains both the volume the beam passed through and the volume where something reflected it, and many readings accumulate into a map of empty and occupied regions. Its defining property for a planner is the third state. A cell that no sensor has yet observed is held as unknown rather than assumed free, which is exactly the distinction a representation optimised for rendering cannot make., combining readings from different sensors and robot positions. Each reading represented a probably empty beam interior and uncertainty about the reflecting point somewhere near the measured range. Overlapping readings refined these constraints Moravec 1985↓High Resolution Maps from Wide Angle SonarHans P. Moravec, Alberto Elfes, ICRA 1985. In their sonar map, a cell with no information is unknown. Zero represents unknown occupancy; negative and positive cell values represent probably empty and probably occupied regions, respectively Moravec 1985↓High Resolution Maps from Wide Angle SonarHans P. Moravec, Alberto Elfes, ICRA 1985.

Signed-distance fields

Curless and Levoy replaced the occupancy value with a distance. Their volumetric method fuses range images into a cumulative weighted signed distance functionsigned-distance fieldA scalar field whose zero crossing represents a surface. Curless and Levoy fuse weighted signed distances measured along sensor lines of sight. KinectFusion uses a projective truncated field rather than a true discrete signed-distance field; fusion approximates a pseudo-Euclidean distance metric. Near the surface, it estimates normals from numerical field derivatives under an orthogonality assumption. It predicts the surface by ray marching and approximate interpolation, not by treating every stored value as an exact collision margin., from which the surface is recovered as the zero crossing Curless 1996↓A Volumetric Method for Building Complex Models from Range ImagesBrian Curless, Marc Levoy, SIGGRAPH 1996. The sign convention must be named. KinectFusion describes positive values toward visible free space and negative values on the non-visible side Newcombe 2011↓KinectFusion: Real-Time Dense Surface Mapping and TrackingRichard A. Newcombe, Shahram Izadi, Otmar Hilliges et al., ISMAR 2011. Curless and Levoy's hole-filling construction instead encodes empty space with negative $D_{min}$ and unseen space with positive $D_{max}$, both at zero weight Curless 1996↓A Volumetric Method for Building Complex Models from Range ImagesBrian Curless, Marc Levoy, SIGGRAPH 1996.

KinectFusion's author-hosted paper describes a Kinect supplying 11-bit, 640×480 depth frames at 30 Hz. Its coarse-to-fine ICP tracks the live depth frame against the fused global model; the commodity-GPU implementation tracks and maps at the sensor frame rate. A large planar scene filling most of the field of view can cause tracking drift or failure Newcombe 2011↓KinectFusion: Real-Time Dense Surface Mapping and TrackingRichard A. Newcombe, Shahram Izadi, Otmar Hilliges et al., ISMAR 2011.

Curless and Levoy truncate the distance ramps and weights near observed range points Curless 1996↓A Volumetric Method for Building Complex Models from Range ImagesBrian Curless, Marc Levoy, SIGGRAPH 1996. KinectFusion uses a projective TSDF: visible-space values beyond the truncation distance $\mu$ are capped at $\mu$, while non-visible points farther than $\mu$ from the surface are not measured Newcombe 2011↓KinectFusion: Real-Time Dense Surface Mapping and TrackingRichard A. Newcombe, Shahram Izadi, Otmar Hilliges et al., ISMAR 2011. KinectFusion distinguishes its projective TSDF from a true discrete signed-distance field. Its projective measurement is correct exactly at the surface or for an isolated point measurement; fusion produces an approximate pseudo-Euclidean distance metric. Near the zero level set, the method assumes the field gradient is orthogonal to the surface and estimates normals using numerical derivatives. Surface prediction marches along each ray to a zero crossing, then approximates the intersection using trilinearly interpolated field values Newcombe 2011↓KinectFusion: Real-Time Dense Surface Mapping and TrackingRichard A. Newcombe, Shahram Izadi, Otmar Hilliges et al., ISMAR 2011.

Meshes, and then neural fields

A mesh adds connectivity, which buys a normal per face and a renderable surface, and costs you a hole wherever nothing was observed. Filling that hole invents geometry, which is fine for a rendering and a problem for a gripper.

NeRF fits a separate radiance field to a static scene. Its MLP predicts density from position and colour from position and viewing direction. Classical volume rendering accumulates those predictions along camera rays, and differentiability lets image error drive optimisation. The inputs include RGB photographs, camera poses, intrinsics and scene bounds; differentiability is not a guarantee that the recovered geometry is correct Mildenhall 2020↓NeRF: Representing Scenes as Neural Radiance Fields for View SynthesisBen Mildenhall, Pratul P. Srinivasan, Matthew Tancik et al., ECCV 2020. The neural radiance fieldneural radiance fieldA scene stored as a continuous function, realised as a small neural network, from a 3D position and a viewing direction to a volume density and a view-dependent colour. Mildenhall and colleagues introduced it: images are synthesised by classical volume rendering along camera rays, and because that rendering is differentiable the whole representation can be optimised from posed photographs alone. The objective is view synthesis, so the geometry it recovers is whatever explains the images rather than whatever a contact solver would want. line then bought three separate things in three steps.

First, view-synthesis quality, which is what the original result demonstrated. Second, lower optimisation cost: Instant NGP combines a trainable multiresolution hash encoding, smaller networks and fully fused CUDA kernels. Its NeRF model still learns from posed RGB images through a differentiable ray marcher; the speedup is not due to the encoding alone. Rendering settings matter too: Figure 12 reports a modular-synthesizer image accumulated over 128 samples in 5 seconds at 1080p on one RTX 3090, and a separate large natural 360-degree scene running at 10 frames per second on the same GPU. Those are two rendering settings, not a robot reconstruction or control rate Müller 2022↓Instant Neural Graphics Primitives with a Multiresolution Hash EncodingThomas Müller, Alex Evans, Christoph Schied et al., ACM Trans. Graph. (SIGGRAPH 2022). Third, rasterisation without a neural network in the rendering loop. 3D Gaussian splatting3D Gaussian splattingA scene stored as a cloud of anisotropic 3D Gaussians, each carrying a position, a covariance, an opacity and a view-dependent colour, rendered by projecting them to the image plane and rasterising rather than by marching rays. Kerbl and colleagues showed the representation reaches radiance-field quality at real-time frame rates, which is what took the family out of offline rendering. The representation has no surface: opacity is a rendering weight, so contact geometry exists only after a separate surface extraction step. optimises Gaussian positions, anisotropic covariances, opacities and spherical-harmonic appearance from static-scene photographs with calibrated cameras. It projects the Gaussians to the image plane and alpha-blends sorted splats. The paper evaluates novel-view quality and rendering speed on established datasets using an A6000 GPU. Its abstract reports real-time rendering at 1080p, while the evaluation retains each dataset's native image resolution; those are not a single fixed-resolution benchmark Kerbl 2023↓3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler et al., ACM Trans. Graph. 42(4), author manuscript (2023).

DUSt3R's network predicts dense pointmaps from a pair of RGB images without camera intrinsics or poses supplied at inference. The pointmaps share the first image's coordinate frame and have an unknown scale. The network is trained with geometric supervision and starts from pretrained CroCo weights. For an image collection, the pipeline then optimises the alignment of pairwise pointmaps in 3D; the feed-forward pairwise prediction is not an optimisation-free multiview reconstruction Wang 2024↓DUSt3R: Geometric 3D Vision Made EasyShuzhe Wang, Vincent Leroy, Yohann Cabon et al., CVPR 2024.

MASt3R extends the same network with a dense local-feature head so matching is grounded in the same 3D prediction Leroy 2024↓Grounding Image Matching in 3D with MASt3RVincent Leroy, Yohann Cabon, Jérôme Revaud, ECCV 2024.

A radiance field is geometry for rendering

This wiki has already staked out the verdict, in the world-models taxonomy: a splat reconstruction is a learned renderer, so appearance is learned and physics is not, and a renderer alone has no answer to what happens under an action. The same conclusion follows from the other direction here. In the original 3DGS model, learned opacity participates in image-space alpha compositing; it is not a calibrated free/occupied/unknown cell classification. The stored primitive is a volumetric Gaussian, not an explicit triangle surface carrying contact normals. That does not make surface extraction impossible: the paper explicitly leaves mesh reconstruction as future work Kerbl 2023↓3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler et al., ACM Trans. Graph. 42(4), author manuscript (2023).

That makes the appropriate uses specific rather than general. SplatSim replaces mesh primitives in the rendering pipeline with Gaussian splats; PyBullet still supplies the physics. The authors report zero-shot real-world transfer of diffusion policies trained on simulated demonstrations in four tasks using a UR5 and Robotiq 2F-85 gripper. Scene preparation includes manual robot segmentation, CAD-derived link bounds and ICP alignment. Figure 2 lists RGB observations plus end-effector position and orientation as policy inputs, while Section IV-A describes solely RGB at test time. The paper gives both descriptions. Its experiments concern rigid-body manipulation Qureshi 2024↓SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian SplattingMohammad Nomaan Qureshi, Sparsh Garg, Francisco Yandun et al., 2024.

RoboGSim combines a Gaussian Reconstructor, Digital Twins Builder, Scene Composer and Interactive Engine. Multi-view images and supplied robot MDH parameters feed reconstruction; mesh assets and measured layout alignment connect the reconstruction to Isaac Sim. In its closed-loop evaluator, a policy acts on splat-rendered images, Isaac Sim handles inverse kinematics and physical interactions, and the resulting state drives the next rendering Li 2024↓RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting SimulatorXinhai Li, Jialin Li, Ziheng Zhang et al., 2024. Novel-view augmentation, and scene memory. A rendering representation alone does not specify a contact model; geometric conversion and contact modelling are separate requirements.

Dex-NeRF is the honest bridge and worth reading as the exception that proves the shape of the rule. It uses a radiance field for geometry rather than for pictures, exploiting the view-independent learned density to render transparency-aware depth, then feeds that depth to a grasp planner. In their RealSense comparison, the camera failed to compute depth for most transparent objects in the tested scenes Ichnowski 2021↓Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent ObjectsJeffrey Ichnowski, Yahav Avigal, Justin Kerr et al., CoRL 2021. The field earned its place by producing a depth map, which is to say by being converted into the kind of representation a planner already knew how to use.

The symbolic end of the ladder deserves a mention for the same reason. OccWorld predicts how a 3D occupancy grid evolves rather than how pixels do, and gets competitive planning results without instance or map supervision Zheng 2023↓OccWorld: Learning a 3D Occupancy World Model for Autonomous DrivingWenzhao Zheng, Weiliang Chen, Yuanhui Huang et al., 2023. Occupancy, again, because occupancy is the thing a planner can act on.

SLAM: the system that builds the map

Every representation above assumes somebody produced it, from sensor data, while the robot was moving and did not know precisely where it was. That is simultaneous localisation and mapping, and state estimation supplies its machinery without ever assembling the system.

In Section II, Cadena and colleagues separate a sensor-dependent front end, which abstracts measurements into models suitable for estimation and associates observations with state variables, from a back end that performs inference on those abstractions. The back end can feed information back to support loop-closure detection and validation Cadena 2016↓Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception AgeCesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016.

Front ends divide into feature-based and direct. A feature-based front end detects repeatable keypoints, describes them, and matches descriptors between frames; the 2015 monocular ORB-SLAM system uses ORB features for tracking, local mapping, relocalisation and loop closing. Tracking matches features and refines camera pose; mapping triangulates matched ORB features and runs local bundle adjustment. A DBoW2 vocabulary built from ORB descriptors supplies relocalisation and loop candidates, which require geometric validation Mur-Artal 2015↓ORB-SLAM: A Versatile and Accurate Monocular SLAM SystemRaúl Mur-Artal, J. M. M. Montiel, Juan D. Tardós, IEEE Transactions on Robotics, 2015.

ORB-SLAM3 supports visual SLAM with monocular, stereo and RGB-D inputs, and monocular-inertial and stereo-inertial SLAM with pinhole or fisheye cameras. Its IMU initialization uses MAP estimation and refinement. Tracking loss first triggers recovery attempts; persistent loss can start a new active map, while early loss after inertial initialization can discard an immature map. A revisit can trigger map merging only after place-recognition candidates pass geometric and covisibility checks; mature visual-inertial maps also undergo gravity-direction checks. Low texture remains a failure case, and slow motion can leave inertial initialization poorly constrained. Campos 2021↓ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAMCarlos Campos, Richard Elvira, Juan J. Gómez Rodríguez et al., IEEE Transactions on Robotics, 2021

A direct front end skips the detector entirely and optimises photometric error over sampled pixels jointly with the geometry, which is Direct Sparse Odometry's formulation Engel 2018↓Direct Sparse OdometryJakob Engel, Vladlen Koltun, Daniel Cremers, IEEE TPAMI, 2018. The trade is legible: features are robust to brightness change and throw away most of the image, while direct methods use weakly textured regions a detector would ignore, and gain accuracy and robustness from a calibration of the camera's exposure, vignetting and response.

Cadena and colleagues describe odometric drift and explain that revisiting mapped landmarks can reduce and possibly correct trajectory drift; they also distinguish wheel odometry from lower-drift visual-inertial methods Cadena 2016↓Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception AgeCesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016. In monocular ORB-SLAM, loop closureloop closureRecognising a previously mapped place and adding a validated constraint between its observations. Cadena and colleagues explain that revisiting mapped landmarks can reduce and possibly correct odometric drift, whose magnitude depends on the odometry method. In monocular ORB-SLAM, similarity alignment and Essential Graph optimization distribute loop-closing error along the graph. This is not a guarantee that every revisit removes all error. requires geometric validation, then uses similarity alignment and Essential Graph optimization to distribute loop-closing error along the graph Mur-Artal 2015↓ORB-SLAM: A Versatile and Accurate Monocular SLAM SystemRaúl Mur-Artal, J. M. M. Montiel, Juan D. Tardós, IEEE Transactions on Robotics, 2015. This is not a guarantee that every revisit removes all error. For visual navigation, place recognitionplace recognitionFor visual navigation, deciding whether the current image depicts a place already in the map. Motion information can also inform that belief. Lowry and colleagues survey the difficulty: the same place can change appearance while different places can look alike. In Cadena and colleagues' robustness argument, false-positive data associations can lead to wrong back-end estimates; false negatives discard useful measurements and reduce estimation accuracy. Geometric verification and robust estimation can mitigate outliers, but do not establish a universal ranking of every false match against every missed match. asks whether the current image matches a place already stored in the map; Lowry and colleagues note that motion information can also inform this belief. Viewpoint, illumination and seasonal changes can alter the same place’s appearance, while different places can look alike (perceptual aliasing) Lowry 2016↓Visual Place Recognition: A SurveyStephanie Lowry, Niko Sünderhauf, Paul Newman et al., IEEE Trans. Robotics, 2016.

In Cadena and colleagues' robustness argument, perceptual aliasing can produce false-positive data associations that lead to wrong back-end estimates; false negatives instead discard useful measurements and reduce estimation accuracy. Robust losses and loop-closure validation can mitigate outliers, but the survey describes remaining failures that depend on initialization and subsequent outlier rejection. This is not a universal ranking of the cost of every false match against every missed match. Cadena 2016↓Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception AgeCesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016

The back end is where the smoothing formulations from state estimation do their work. Square Root SAM jointly estimates the trajectory and landmark map by maximum a posteriori (MAP) inference. Under Gaussian process and measurement models with known data associations, this becomes a nonlinear least-squares problem. With the reference frame fixed, nonlinear optimisation solves successive sparse linearised systems using QR of the measurement Jacobian or Cholesky of the information matrix; variable ordering controls fill-in Dellaert 2006↓Square Root SAM: Simultaneous Localization and Mapping via Square Root Information SmoothingFrank Dellaert, Michael Kaess, Int. J. Robotics Research, 2006. iSAM2 uses a Bayes tree for incremental reordering and relinearisation: it re-eliminates affected cliques and their ancestors, then reattaches unaffected subtrees. Thresholded state updates trade some accuracy for speed, and large loop closures can cost as much as a batch solve Kaess 2012↓iSAM2: Incremental Smoothing and Mapping Using the Bayes TreeMichael Kaess, Hordur Johannsson, Richard Roberts et al., Int. J. Robotics Research, 2012. Loop closure is precisely why smoothing displaced filtering here: relinearising the past is only possible if the past is still in the graph.

Cadena and colleagues describe sparse landmark maps built from discriminative features and dense geometric models better suited to obstacle avoidance and planning; dense models also appear in visual SLAM. They assess representations by storage size, construction cost and usefulness for the task, and describe more efficient volumetric storage schemes such as octrees and voxel hashing Cadena 2016↓Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception AgeCesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016.

Sensor choice depends on the sensor-environment pair: Cadena and colleagues note that two rooms may look identical to a 2D laser scanner yet be distinguishable by appearance to a camera. Severe illumination changes defeat the bag-of-words methods discussed in the review, while active range cameras can work in dark, untextured scenes but face range and external-light limitations Cadena 2016↓Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception AgeCesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016.

iMAP learns a scene-specific MLP from scratch from a handheld RGB-D stream, without prior training data. The network maps 3D coordinates to colour and volume density; rendering uses camera intrinsics. Tracking optimises the current camera pose against a fixed network, while mapping updates the network and selected keyframe poses. The MLP is the dense scene map, not the system's only stored data: RGB-D keyframes and poses are retained for replay, and network snapshots guide keyframe selection. The paper reports tracking at 10 Hz and mapping at 2 Hz on a single desktop CPU/GPU system. Live operation drops images arriving between processed frames; fine map detail takes minutes to develop. The demonstrated scope is room-scale mapping, and meshes are extracted for evaluation and visualisation rather than used by SLAM Sucar 2021↓iMAP: Implicit Mapping and Positioning in Real-TimeEdgar Sucar, Shikun Liu, Joseph Ortiz et al., ICCV 2021.

NICE-SLAM uses hierarchical feature grids with three fixed, pretrained geometry decoders for coarse-, mid- and fine-scale occupancy, plus a separate colour grid and a colour decoder optimised online. The geometry decoders are pretrained using ConvONet's Synthetic Indoor Scene Dataset; scene grids are then optimised from RGB-D observations, with camera intrinsics used for rendering. Tracking estimates the current camera pose; the static-scene mapping formulation performs local bundle adjustment over scene features, the colour decoder and selected keyframe poses. The paper demonstrates reconstruction of a self-captured multi-room apartment, rather than unlimited scene scale. Geometric updates are local, but a global keyframe list is retained, colour can suffer forgetting and remain consistent only locally, and the system does not perform loop closure. Its reported 47 ms tracking and 130 ms mapping timings use 200 tracking pixels and 1,000 mapping pixels on an RTX 3090/Intel i7-10700K desktop. The large-scene experiments use higher sample counts, so those timings are not a latency guarantee for the apartment Zhu 2022↓NICE-SLAM: Neural Implicit Scalable Encoding for SLAMZihan Zhu, Songyou Peng, Viktor Larsson et al., CVPR 2022.

The deployed answer: costmaps and two planners

Navigation2 provides one concrete mobile-robot implementation of this pipeline. The map a navigation stack plans over is a costmapcostmapThe grid a mobile-robot navigation stack plans over: occupancy from the map and the live sensors, inflated by the robot's footprint and marked up with whatever else should influence the route, so a planner searching for a cheap path is also searching for a safe one. Lu, Hershberger and Smart proposed and implemented layered costmaps in ROS Navigation: ordered semantic layers update a master 2D costmap, bounds first, then values. Layers may keep private grids or write directly to the master. Sensed obstacles may overwrite static-map costs if configured.: occupancy from the map and the live sensors, inflated by the robot's footprint, marked up with whatever else should influence the route. Lu, Hershberger and Smart proposed and implemented layered costmaps in the ROS Navigation stack. The method keeps an ordered list of semantically separate layers and accumulates their data into a master 2D costmap for path planning. Each update first gathers the layers' update bounds, then lets each layer update the master grid within those bounds. Some layers keep private grids, while others operate directly on the master; separating static-map and sensed-obstacle data does not prohibit overwriting static-map costs, which remains configurable Lu 2014↓Layered Costmaps for Context-Sensitive NavigationDavid V. Lu, Dave Hershberger, William D. Smart, IROS 2014. Navigation2 uses a layered costmap Macenski 2020↓The Marathon 2: A Navigation SystemSteve Macenski, Francisco Martin, Ruffin White et al., IROS 2020.

Navigation2 separates global planning from local control: the global planner computes a shortest route to a goal, while the controller uses local information to compute a local path and control signals. The two are plugins delegated to task-specific asynchronous servers. Navigation2, built on ROS 2, is the implementation used as the example here Macenski 2020↓The Marathon 2: A Navigation SystemSteve Macenski, Francisco Martin, Ruffin White et al., IROS 2020. The algorithms inside both are the ones in motion planning, applied to a 2D grid rather than a robot arm's configuration space.

Notice what the deployed stack chose. Not the representation that renders best, and not the one with the most parameters, but the one that answers the question the planner asks, which is whether a cell is free.

A representation is a service contract

Choose a scene representation by the queries its consumers issue. Localization needs stable landmarks and uncertainty. Collision checking needs occupied volume, resolution and a conservative update rule. Grasping may need surface pose and object identity. Rendering needs view-dependent appearance. One representation can support several queries, but no reconstruction should be promoted merely because it looks complete.

For deployment, publish the frame, timestamp, resolution, confidence or unknown-state convention, update rate and invalidation rule with the map. Test that attached objects and removed obstacles propagate to the planning view before motion begins. The ROS 2 guide describes this handoff through transforms and planning-scene updates. The distinction from a learned world model is equally important: a scene representation answers what is present now, while a world model must answer how the state changes under an action. The simulator comparison makes that boundary explicit.