Scene Representation and Mapping

Scene representation is how a robot stores what it knows about its surroundings so that planners, controllers and other consumers can query it^{Source 8}8 Cadena et al. 2016, Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age. Cesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016. Representations range from point clouds and occupancy grids to signed-distance fields, meshes and neural fields such as radiance fields and Gaussian splats^{Source 8}8 Cadena et al. 2016, Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age. Cesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016 ^{Source 23}23 Irshad et al. 2024, Neural Fields in Robotics: A Survey. Muhammad Zubair Irshad, Mauro Comi, Yen-Chen Lin et al., 2024.

Photo: Daniel L. Lu / Wikimedia Commons. Licence: CC BY 4.0.

The ladder, and what each rung can answer

Three queries separate the rungs: whether a cell is free for a planner, which way a surface faces for a contact solver, and what the scene looks like from a viewpoint the sensor never occupied. Cadena and colleagues judge representations by storage size, construction cost and usefulness for the task^{Source 8}8 Cadena et al. 2016, Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age. Cesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016.

Point clouds and occupancy grids

A point cloudpoint cloudA point cloud is a set of 3D points, usually with no ordering and no connectivity, as produced by back-projecting depth-camera or lidar measurements through the camera intrinsics. A network reading it must be permutation invariant. PointNet uses a shared per-point encoder and symmetric pooling; PointNet++ recursively applies local PointNets to metric-space neighbourhoods for multi-scale features. Pooling summarizes, so some geometry can be lost; density-adaptive grouping addresses sparse neighbourhoods. is an unordered set of 3D points, and PointNet requires invariance to the order in which they arrive^{Source 22}22 Qi et al. 2017, PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Charles R. Qi, Hao Su, Kaichun Mo et al., CVPR 2017. A region without points is either empty or unobserved, and the store cannot say which.

Moravec and Elfes projected wide-angle sonar readings from different sensors and robot positions onto a 2D occupancy gridoccupancy gridAn occupancy grid is a map of cells, each storing the probability that it is occupied. Moravec and Elfes introduced it for wide-angle sonar, where each reading constrains both the space the beam crossed and where it reflected. Many readings build a map of empty and occupied regions. For a planner, its defining property is a third state: an unobserved cell is marked unknown, which rendering-oriented representations cannot express.. Each reading marked a probably empty beam interior and an uncertain reflecting point near the measured range, and overlapping readings refined these constraints. Zero represented unknown occupancy, and negative and positive values probably empty and probably occupied cells^{Source 1}1 Moravec et al. 1985, High Resolution Maps from Wide Angle Sonar. Hans P. Moravec, Alberto Elfes, ICRA 1985. OctoMap carries the three states into 3D in a compressed octree^{Source 24}24 Hornung et al. 2013, OctoMap: an efficient probabilistic 3D mapping framework based on octrees. Armin Hornung, Kai M. Wurm, Maren Bennewitz et al., Autonomous Robots, 2013. Step through the figure below to see one mug stored four ways.

What does a robot actually see?

A depth camera also measures how far away each point is. Each point becomes a dot, but only on the side it sees.

BackNext

A robot can store a mug as dots, cubes, a skin or soft blobs, and each suits a different job.

How this was made

The steps in words

1. A depth camera also measures how far away each point is. Each point becomes a dot, but only on the side it sees.
2. Cubes split space into a grid. Each box is either part of the mug or empty. Blocky, but simple.
3. A skin joins points on the surface into small flat triangles, giving the smooth, exact shape a robot needs to plan its grip.
4. Soft blobs overlap into one surface. Each blob can also carry colour, so together they can look like a photo.
5. Same mug, four ways to store it. Each trades detail against speed and memory. These are scene representations.

Signed-distance fields

Curless and Levoy fuse range images into a cumulative weighted signed distance functionsigned-distance fieldA signed-distance field is a scalar field whose zero crossing represents a surface. Curless and Levoy fuse weighted signed distances measured along sensor lines of sight. KinectFusion uses a projective truncated field, an approximation of a true discrete signed-distance field; fusion approximates a pseudo-Euclidean distance metric. It estimates near-surface normals from numerical field derivatives, assuming orthogonality. It predicts the surface by ray marching and approximate interpolation, and no stored value is an exact collision margin. and recover the surface at its zero crossing, truncating distance ramps and weights near observed points. Their hole-filling construction marks empty space with negative $D_{min}$ and unseen space with positive $D_{max}$, both at zero weight^{Source 2}2 Curless et al. 1996, A Volumetric Method for Building Complex Models from Range Images. Brian Curless, Marc Levoy, SIGGRAPH 1996. KinectFusion uses the opposite sign: positive toward visible free space and negative on the non-visible side^{Source 3}3 Newcombe et al. 2011, KinectFusion: Real-Time Dense Surface Mapping and Tracking. Richard A. Newcombe, Shahram Izadi, Otmar Hilliges et al., ISMAR 2011.

KinectFusion fused 11-bit, 640×480 Kinect depth frames at 30 Hz, tracking each frame against the global model by coarse-to-fine ICP and mapping at the sensor rate on a commodity GPU. A large plane filling most of the view can cause drift or tracking failure. Its projective TSDF caps visible-space values at the truncation distance $\mu$, leaves non-visible points beyond $\mu$ unmeasured and is exact at the surface, so fusion yields an approximate pseudo-Euclidean distance. Normals come from numerical derivatives near the zero level set, and ray marching finds surfaces by trilinear interpolation^{Source 3}3 Newcombe et al. 2011, KinectFusion: Real-Time Dense Surface Mapping and Tracking. Richard A. Newcombe, Shahram Izadi, Otmar Hilliges et al., ISMAR 2011.

Photo: Antonio Roberts / Wikimedia Commons. Licence: CC BY-SA 2.0.

Trajectory optimizers need obstacle distances. Voxblox builds TSDFs faster than OctoMap builds its maps, then derives Euclidean signed distance fields (ESDFs) from them incrementally, more accurately than from occupancy^{Source 25}25 Oleynikova et al. 2016, Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planning. Helen Oleynikova, Zachary Taylor, Marius Fehr et al., arXiv preprint (IROS 2017), 2016, and on GPUs nvblox reports up to 177x faster surface reconstruction and 31x faster distance-field computation^{Source 26}26 Millane et al. 2023, nvblox: GPU-Accelerated Incremental Signed Distance Field Mapping. Alexander Millane, Helen Oleynikova, Emilie Wirbel et al., arXiv preprint (ICRA 2024), 2023.

Meshes, and then neural fields

A mesh adds connectivity, giving each face a normal and a renderable surface, but leaves holes where nothing was observed, and filling them invents geometry^{Source 2}2 Curless et al. 1996, A Volumetric Method for Building Complex Models from Range Images. Brian Curless, Marc Levoy, SIGGRAPH 1996.

NeRF fits a neural radiance fieldneural radiance fieldA neural radiance field is a scene stored as a continuous function, a small neural network, from 3D position and viewing direction to volume density and view-dependent colour. Mildenhall et al. introduced it, synthesising images by differentiable volume rendering along camera rays, so the scene can be optimised from posed photographs alone. Its objective is view synthesis, so nothing in training shapes its geometry for a contact solver. to a static scene from posed RGB photographs, intrinsics and scene bounds. Its MLP predicts density from position and colour from position and viewing direction, and differentiable volume rendering lets image error drive optimization^{Source 4}4 Mildenhall et al. 2020, NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik et al., ECCV 2020.

Instant NGP cut optimization cost with a multiresolution hash encoding, smaller networks and fully fused CUDA kernels, and rendered one scene at 1080p in 5 seconds over 128 samples and a large 360-degree scene at 10 frames per second on one RTX 3090^{Source 5}5 Müller et al. 2022, Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. Thomas Müller, Alex Evans, Christoph Schied et al., ACM Trans. Graph. (SIGGRAPH 2022). 3D Gaussian splatting3D Gaussian splatting3D Gaussian splatting is a scene representation of anisotropic 3D Gaussians, each with position, covariance, opacity and view-dependent colour, rasterised onto the image plane instead of ray-marched. Kerbl et al. showed it reaches radiance-field quality at real-time frame rates, taking the family out of offline rendering. It has no surface: opacity is a rendering weight, so contact geometry needs separate surface extraction. optimizes Gaussian positions, anisotropic covariances, opacities and spherical-harmonic appearance from calibrated photographs, then rasterizes by alpha-blending sorted splats. It reports real-time 1080p rendering, with results on an A6000 GPU^{Source 6}6 Kerbl et al. 2023, 3D Gaussian Splatting for Real-Time Radiance Field Rendering. Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler et al., ACM Trans. Graph. 42(4), author manuscript (2023).

DUSt3R predicts dense pointmaps from a pair of RGB images without intrinsics or poses, in the first image's frame and at unknown scale, trained with geometric supervision from pretrained CroCo weights. For an image collection, it then optimizes the alignment of pairwise pointmaps in 3D^{Source 7}7 Wang et al. 2024, DUSt3R: Geometric 3D Vision Made Easy. Shuzhe Wang, Vincent Leroy, Yohann Cabon et al., CVPR 2024.

Feed-forward models now cover whole collections: VGGT infers cameras, point maps, depth maps and point tracks from one, a few or hundreds of views in under one second^{Source 27}27 Wang et al. 2025, VGGT: Visual Geometry Grounded Transformer. Jianyuan Wang, Minghao Chen, Nikita Karaev et al., arXiv preprint (CVPR 2025).

A radiance field is geometry for rendering

The world-models taxonomy treats a splat reconstruction as a learned renderer: appearance is learned and physics is not. In the original 3DGS, learned opacity is an image-space alpha-compositing weight with no free, occupied or unknown label, and the stored primitive is a volumetric Gaussian^{Source 6}6 Kerbl et al. 2023, 3D Gaussian Splatting for Real-Time Radiance Field Rendering. Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler et al., ACM Trans. Graph. 42(4), author manuscript (2023). The paper names mesh reconstruction as an open direction^{Source 6}6 Kerbl et al. 2023, 3D Gaussian Splatting for Real-Time Radiance Field Rendering. Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler et al., ACM Trans. Graph. 42(4), author manuscript (2023), and SuGaR then aligned Gaussians with surfaces and extracted meshes by Poisson reconstruction^{Source 28}28 Guédon et al. 2023, SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering. Antoine Guédon, Vincent Lepetit, 2023.

Simulators pair splats for appearance with separate physics: SplatSim renders with splats while PyBullet supplies physics^{Source 16}16 Qureshi et al. 2024, SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting. Mohammad Nomaan Qureshi, Sparsh Garg, Francisco Yandun et al., 2024. Its authors report zero-shot real-world transfer of diffusion policies on four rigid-body tasks with a UR5 and Robotiq 2F-85 gripper, after manual robot segmentation and ICP alignment^{Source 16}16 Qureshi et al. 2024, SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting. Mohammad Nomaan Qureshi, Sparsh Garg, Francisco Yandun et al., 2024. In RoboGSim's closed-loop evaluator, a policy acts on splat-rendered images while Isaac Sim handles inverse kinematics and physical interaction, from multi-view images, supplied MDH parameters and mesh assets^{Source 17}17 Li et al. 2024, RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting Simulator. Xinhai Li, Jialin Li, Ziheng Zhang et al., 2024.

Other fields serve a planner directly. Dex-NeRF renders transparency-aware depth from learned density for a grasp planner, where a RealSense camera failed to compute depth for most transparent objects tested^{Source 15}15 Ichnowski et al. 2021, Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects. Jeffrey Ichnowski, Yahav Avigal, Justin Kerr et al., CoRL 2021.

ParticleSplat, from Deepak Pathak (Carnegie Mellon) and colleagues (September 2026), encodes posed multi-view images into latent particles that decode to 3D Gaussians, trained with a novel-view synthesis objective. Object masks emerge without labels, moving a particle moves its object, and the representation improved downstream manipulation^{Source 35}35 He et al. 2026, ParticleSplat: Self-supervised Object-centric Latent Particle Splatting. Lyuxing He, Daniel Guo, Elizabeth Terveen et al., 2026.

SLAM: the system that builds the map

Simultaneous localization and mapping (SLAM) builds such maps from sensor data while the robot moves without knowing its exact pose. Cadena and colleagues separate a sensor-dependent front end, which abstracts measurements and associates them with state variables, from a back end that performs inference and feeds information back for loop-closure detection and validation^{Source 8}8 Cadena et al. 2016, Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age. Cesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016. State estimation covers the inference machinery.

Photo: Steve Jurvetson / Wikimedia Commons. Licence: CC BY 2.0.

Front ends are feature-based or direct. Monocular ORB-SLAM (2015) uses ORB features for tracking, local mapping, relocalization and loop closing, triangulating matched features and running local bundle adjustment. Its DBoW2 vocabulary proposes relocalization and loop candidates that need geometric validation^{Source 9}9 Mur-Artal et al. 2015, ORB-SLAM: A Versatile and Accurate Monocular SLAM System. Raúl Mur-Artal, J. M. M. Montiel, Juan D. Tardós, IEEE Transactions on Robotics, 2015.

ORB-SLAM3 supports stereo, RGB-D and visual-inertial modes with pinhole or fisheye cameras and MAP-based IMU initialization. It starts a new map after persistent tracking loss and merges maps only after geometric and covisibility checks, plus gravity-direction checks for mature visual-inertial maps. Low texture remains the main failure case, and slow motion can delay inertial initialization^{Source 10}10 Campos et al. 2021, ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM. Carlos Campos, Richard Elvira, Juan J. Gómez Rodríguez et al., IEEE Transactions on Robotics, 2021.

Direct Sparse Odometry skips the detector and optimizes photometric error over sampled pixels jointly with geometry^{Source 11}11 Engel et al. 2018, Direct Sparse Odometry. Jakob Engel, Vladlen Koltun, Daniel Cremers, IEEE TPAMI, 2018. It can sample any region with intensity gradient, including near-featureless walls, and gains from calibrating exposure, vignetting and camera response^{Source 11}11 Engel et al. 2018, Direct Sparse Odometry. Jakob Engel, Vladlen Koltun, Daniel Cremers, IEEE TPAMI, 2018.

Learned systems keep classical machinery: DROID-SLAM iterates camera pose and per-pixel depth through a dense bundle-adjustment layer^{Source 29}29 Teed et al. 2021, DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras. Zachary Teed, Jia Deng, 2021.

Revisiting mapped landmarks can reduce odometric drift and in some cases correct it, and visual-inertial methods drift less than wheel odometry^{Source 8}8 Cadena et al. 2016, Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age. Cesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016. Monocular ORB-SLAM validates a loop closureloop closureLoop closure is recognising a mapped place and adding a validated constraint between observations. Cadena and colleagues note revisiting mapped landmarks can reduce, and possibly correct, odometric drift, whose size depends on the odometry method. In monocular ORB-SLAM, similarity alignment and Essential Graph optimization spread loop-closing error along the graph, so a revisit can leave residual error. geometrically, then spreads its error with similarity alignment and Essential Graph optimization^{Source 9}9 Mur-Artal et al. 2015, ORB-SLAM: A Versatile and Accurate Monocular SLAM System. Raúl Mur-Artal, J. M. M. Montiel, Juan D. Tardós, IEEE Transactions on Robotics, 2015.

Place recognitionplace recognitionPlace recognition is deciding, in visual navigation, whether the current image shows a mapped place. Motion information can also inform that belief. Lowry et al. survey why it is hard: places change appearance, and different places can look alike. In Cadena and colleagues' robustness argument, false-positive data associations can produce wrong back-end estimates; false negatives discard useful measurements and reduce estimation accuracy. Geometric verification and robust estimation can mitigate outliers, but do not establish a universal ranking of every false match against every missed match. asks whether the current image matches a stored place, and motion information can also inform that belief. Viewpoint, illumination and seasonal changes alter one place's appearance, and different places can look alike, which is perceptual aliasing^{Source 12}12 Lowry et al. 2016, Visual Place Recognition: A Survey. Stephanie Lowry, Niko Sünderhauf, Paul Newman et al., IEEE Trans. Robotics, 2016.

Perceptual aliasing can create false-positive associations that corrupt back-end estimates, while false negatives discard useful measurements. Robust losses and loop-closure validation mitigate outliers, though failures remain that depend on initialization^{Source 8}8 Cadena et al. 2016, Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age. Cesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016.

In the back end, Square Root SAM estimates the trajectory and landmark map jointly by MAP inference, which under Gaussian models with known data associations is sparse nonlinear least squares. It solves successive linearized systems by QR or Cholesky, and variable ordering controls fill-in^{Source 18}18 Dellaert et al. 2006, Square Root SAM: Simultaneous Localization and Mapping via Square Root Information Smoothing. Frank Dellaert, Michael Kaess, Int. J. Robotics Research, 2006. iSAM2 adds incremental reordering and relinearization on a Bayes tree. Thresholded updates trade some accuracy for speed, and a large loop closure can cost as much as a batch solve^{Source 19}19 Kaess et al. 2012, iSAM2: Incremental Smoothing and Mapping Using the Bayes Tree. Michael Kaess, Hordur Johannsson, Richard Roberts et al., Int. J. Robotics Research, 2012.

Smoothing suits loop closure because relinearizing the past requires the past to stay in the graph. Cadena and colleagues contrast sparse landmark maps with dense models suited to obstacle avoidance, and cite octrees and voxel hashing as efficient volumetric storage. Two rooms can look identical to a 2D laser yet differ to a camera. Severe illumination changes defeat bag-of-words methods, and active range cameras work in dark, untextured scenes but face range and external-light limits^{Source 8}8 Cadena et al. 2016, Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age. Cesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016.

The iMAP system learns a scene-specific MLP from scratch at room scale from a handheld RGB-D stream. It tracks at 10 Hz and maps at 2 Hz on a desktop CPU and GPU, retaining keyframes for replay^{Source 13}13 Sucar et al. 2021, iMAP: Implicit Mapping and Positioning in Real-Time. Edgar Sucar, Shikun Liu, Joseph Ortiz et al., ICCV 2021. NICE-SLAM uses hierarchical feature grids with pretrained geometry decoders, reconstructs a multi-room apartment and performs no loop closure^{Source 14}14 Zhu et al. 2022, NICE-SLAM: Neural Implicit Scalable Encoding for SLAM. Zihan Zhu, Songyou Peng, Viktor Larsson et al., CVPR 2022. Its 47 ms tracking and 130 ms mapping times use small pixel samples on an RTX 3090^{Source 14}14 Zhu et al. 2022, NICE-SLAM: Neural Implicit Scalable Encoding for SLAM. Zihan Zhu, Songyou Peng, Viktor Larsson et al., CVPR 2022.

SplaTAM maps with 3D Gaussians from an RGB-D camera and uses a silhouette mask to determine whether areas were previously mapped^{Source 30}30 Keetha et al. 2023, SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM. Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula et al., arXiv preprint (CVPR 2024), 2023.

The deployed answer: costmaps and two planners

A navigation stack plans over a costmapcostmapA costmap is the grid a mobile-robot navigation stack plans over: map and live sensor occupancy, inflated by the robot's footprint and marked with other route costs. A planner seeking a cheap path thus also seeks a safe one. Lu, Hershberger and Smart proposed and implemented layered costmaps in ROS Navigation: ordered semantic layers update a master 2D costmap, bounds first, then values. Layers may keep private grids or write directly to the master. Sensed obstacles may overwrite static-map costs if configured.: occupancy from the map and live sensors, inflated by the robot's footprint. Lu, Hershberger and Smart's layered costmaps keep an ordered list of semantically separate layers and accumulate them into a master 2D costmap. Each update gathers the layers' bounds, then lets each layer write the master grid within them, and a configuration setting decides whether sensed obstacles overwrite static-map costs^{Source 20}20 Lu et al. 2014, Layered Costmaps for Context-Sensitive Navigation. David V. Lu, Dave Hershberger, William D. Smart, IROS 2014.

Navigation2, built on ROS 2, uses a layered costmap. Its global planner computes a shortest route to the goal, and a separate controller computes a local path and control signals, each a plugin in an asynchronous server^{Source 21}21 Macenski et al. 2020, The Marathon 2: A Navigation System. Steve Macenski, Francisco Martin, Ruffin White et al., IROS 2020.

The layered costmap's track_unknown_space option defaults to false, which treats unknown space as free; keeping the unknown state requires enabling it^{Source 31}31 Nav2 Project 2026, Costmap 2D. Nav2 Project, Nav2 Documentation (Lyrical), as of 2026-10-04. The planning algorithms are those of motion planning on a 2D grid.

A representation is a service contract

Perception locates objects in the current frame^{Source 33}33 Gu et al. 2023, ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning. Qiao Gu, Alihusein Kuwajerwala, Sacha Morin et al., 2023. Consumers issue different queries: localization needs stable landmarks^{Source 8}8 Cadena et al. 2016, Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age. Cesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016, collision checking needs occupied volume and a conservative update rule^{Source 24}24 Hornung et al. 2013, OctoMap: an efficient probabilistic 3D mapping framework based on octrees. Armin Hornung, Kai M. Wurm, Maren Bennewitz et al., Autonomous Robots, 2013, grasping needs surface pose and identity, and rendering needs appearance^{Source 8}8 Cadena et al. 2016, Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age. Cesar Cadena, Luca Carlone, Henry Carrillo et al., IEEE Transactions on Robotics, 2016.

Hydra stacks rungs in one real-time system, building a local ESDF, extracting a topological map of places from it and segmenting places into rooms^{Source 32}32 Hughes et al. 2022, Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization. Nathan Hughes, Yun Chang, Luca Carlone, Robotics: Science and Systems (RSS) 2022.

SceneLM, from Jitendra Malik (UC Berkeley), Marco Pavone and colleagues (September 2026), keeps the whole map as a structured text list of objects that one vision-language model edits after each image. On language-grounded retrieval and localization benchmarks it performed competitively with complete mapping pipelines, with a map 6 to 12 times more compact, and it ran online on an edge device aboard a quadruped^{Source 36}36 Lilja et al. 2026, A Scene Language Model for Open-Vocabulary Scene Mapping. Adam Lilja, Fabio Hübel, Siming He et al., 2026.

Ledger, from Shravan Chaudhari, Homanga Bharadhwaj and colleagues at Johns Hopkins (October 2026), keeps a persistent 3D object memory from a person's egocentric video: each object's resting locations, its moves and a short description, retained after it leaves view. A language model answering from these records as text, without the images, raised GPT-5.4's HD-EPIC accuracy from 29.7% to 42.6%, and on Ego4D the memory located objects with a median error of 0.99 m where it returned a prediction^{Source 37}37 Chaudhari et al. 2026, Never Look Back: Understanding Persistence in 3D Object Memory from Egocentric Videos. Shravan Chaudhari, William Paul, Suchi Saria et al., 2026.

A deployed map publishes its frame, timestamp, resolution, unknown-state convention, update rate and invalidation rule. In MoveIt 2, the planning scene holds the robot's current state, its model and the world representation for collision checking^{Source 34}34 MoveIt Maintainers 2026, Planning Scene. MoveIt Maintainers, MoveIt 2 Documentation, as of 2026-08-24, and attached or removed objects must reach it before motion; the ROS 2 guide covers transforms and the planning scene. A scene representation answers what is present now, while a world model must answer how state changes under an action, as the simulator comparison sets out.