Action Spaces for Robot Learning

An action space is the interface between a policy and the rest of the robot. It decides what the network predicts, which errors the low-level controller must absorb, how frequently inference must run, and whether data from two embodiments can share labels. Calling both outputs “actions” hides differences large enough to reverse a model comparison.

The useful question is not which action space is best. It is which responsibilities should live in the learner and which should remain in a controller with explicit geometry, dynamics and safety limits.

Try the check below and choose the Cartesian interface only if the controller underneath can uphold its side of the contract.

Self-check

A policy must transfer between two 6-DoF arms with different link lengths, while each robot already has a stable Cartesian controller. Which first action representation is the cleanest shared interface?

Absolute joint positions for each robotRelative Cartesian end-effector deltas plus gripper stateRaw motor torques at the policy inference rate

Read the reasoning

• Absolute joint positions for each robotThe same joint vector places different robots in different physical poses. This is easy to replay on one embodiment and a poor shared task-space label across different kinematic chains.
• Relative Cartesian end-effector deltas plus gripper stateThis expresses motion where the task happens and delegates embodiment-specific inverse kinematics and tracking to each robot controller. The frame and rotation convention still have to be identical.
• Raw motor torques at the policy inference rateTorque can express contact behavior, but it is tightly coupled to dynamics, actuator limits and a high-rate safety loop. It is the least portable starting point for these two arms.

Cross-embodiment transfer improves when the label describes task-space motion and each robot owns the kinematic conversion underneath.

The contract has four parts

An action is not only a vector. A complete action contract specifies:

1. Coordinates: joint, Cartesian, object-relative or another frame.
2. Semantics: absolute target, delta, velocity, force, torque or controller parameter.
3. Timing: control rate, prediction horizon and how stale predictions are handled.
4. Execution: interpolation, clipping, inverse kinematics, feedback control and safety limits.

Two papers can both report a seven-dimensional action while meaning different things. One may emit an end-effector translation and axis-angle rotation delta at 10 Hz, integrated by an operational-space controller. Another may emit absolute joint targets at 50 Hz. The output widths match; the learning problems do not.

Joint-space actions

Joint position targets are the common imitation-learning baseline. They align naturally with leader-follower teleoperation: the demonstrated joint angles can become the labels with little transformation. ACT used continuous targets for two six-joint ViperX arms plus their grippers, predicted as chunks rather than one step at a time Zhao 2023↓Learning Fine-Grained Bimanual Manipulation with Low-Cost HardwareTony Z. Zhao, Vikash Kumar, Sergey Levine et al., RSS 2023.

The advantage is repeatability on one mechanism. The controller can interpolate targets, enforce joint limits and close a high-rate loop below the policy. The cost is embodiment specificity. A shoulder angle on one arm does not describe the same end-effector motion on another.

Joint velocity targets remove the dependence on an absolute zero and can make short-horizon corrections smoother. They drift when integrated, and the same velocity still maps to different Cartesian motion under different link geometries.

Both representations require an accurate kinematic model. The kinematics module derives the forward map and Jacobian that connect joint motion to the hand Lynch 2017↓Modern Robotics: Mechanics, Planning, and ControlKevin M. Lynch, Frank C. Park, Cambridge University Press, 2017.

Cartesian actions

Cartesian actions describe what the tool should do: translate, rotate, open or close. Relative deltas are popular because they keep values locally bounded and reduce dependence on an absolute world origin. They also provide a plausible shared label for arms whose joint counts and link lengths differ.

The simplicity is above the waterline. Underneath, each robot needs inverse kinematics or an operational-space controller, singularity handling, joint-limit avoidance and a named reference frame. Khatib's operational-space formulation is the classical statement of controlling motion and force in task coordinates rather than joint coordinates Khatib 1987↓A Unified Approach for Motion and Force Control of Robot Manipulators: The Operational Space FormulationOussama Khatib, IEEE J. Robotics and Automation, 1987.

Rotation needs special care. Euler-angle deltas have discontinuities and depend on axis order. Axis-angle and exponential coordinates are compact but must define whether the increment is expressed in the tool or world frame. A six-number action is not portable until that convention is written down.

NVIDIA's GR00T N1.7 moved to a shared relative end-effector action space to align egocentric human motion with robot motion NVIDIA 2026↓NVIDIA Isaac GR00TNVIDIA, 2026. That is a representation choice, not merely a preprocessing detail: it determines which non-robot data can become a useful target.

Torque and impedance actions

Torque actions give a policy direct authority over dynamics and contact. They can express compliant insertion, balancing and fast reactions that position targets hide. They also demand a high control rate, accurate timing, actuator-specific scaling and a safety layer that remains effective when the learned output is wrong. Franka's control interface exposes a 1 kHz torque-level loop, which illustrates the rate at which this contract normally lives Franka Robotics 2026↓Franka Control Interface DocumentationFranka Robotics, Franka Robotics, as of 2026-08-20.

Impedance actions are a useful middle layer. The policy selects a pose target plus stiffness and damping, while a controller converts the error into forces. Variable end-effector impedance has been studied explicitly as an RL action space for contact-rich tasks Martin-Martin 2019↓Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich TasksRoberto Martin-Martin, Michelle A. Lee, Rachel Gardner et al., IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2019. It gives learning control over compliance without asking it to synthesize every torque sample.

This split is often the engineering sweet spot: the learner decides how the robot should yield, and a verified controller owns the fast feedback.

Chunks change the time axis

An action chunkaction chunkingPredicting a sequence of future actions in one inference instead of a single action per timestep. Introduced with ACT on the ALOHA platform, chunking cuts the number of sequential decisions in an episode by the chunk length, which reduces compounding error and gives the policy the temporal context it needs to represent pauses, at the cost of committing to a plan between replans. predicts several future steps per inference. In ACT's ablations, chunking improved ACT and the two baselines the authors retrofitted with it, and the CVAE objective proved essential on human demonstrations: removing it cut success on the two simulated human-data tasks from 35.3% to 2% Zhao 2023↓Learning Fine-Grained Bimanual Manipulation with Low-Cost HardwareTony Z. Zhao, Vikash Kumar, Sergey Levine et al., RSS 2023. Diffusion Policy also predicts a horizon and executes only the first portion before replanning Chi 2023↓Diffusion Policy: Visuomotor Policy Learning via Action DiffusionCheng Chi, Zhenjia Xu, Siyuan Feng et al., 2023.

Chunks reduce inference pressure and model temporally coordinated motion. They also create a stale-plan problem. If contact makes the world diverge after the first few steps, the remaining chunk reflects an observation that is no longer current. Receding-horizon execution, temporal ensembling and real-time chunk replacement Black 2025↓Real-Time Execution of Action Chunking Flow PoliciesKevin Black, Manuel Y. Galliker, Sergey Levine, 2025 are different answers to that problem.

Report the chunk horizon and the executed horizon separately. “Predicts 16 actions” does not tell a reader whether all 16 are sent open loop or whether the policy replans after four.

Tokens change the output geometry

RT-2 Brohan 2023↓RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic ControlAnthony Brohan, Noah Brown, Justice Carbajal et al., 2023 and OpenVLA mapped continuous action dimensions into discrete vocabulary tokens so robot control could use language-model training machinery. OpenVLA discretizes seven action dimensions into 256 bins and emits the tokens autoregressively Kim 2024↓OpenVLA: An Open-Source Vision-Language-Action ModelMoo Jin Kim, Karl Pertsch, Siddharth Karamcheti et al., 2024. This makes cross-entropy training easy and adds quantization error plus sequential decoding latency.

FAST takes the opposite route: transform an action chunk into frequency coefficients, quantize the sparse coefficients, then apply byte-pair encoding. Its universal FAST+ tokenizer was trained across one million robot trajectories Pertsch 2025↓FAST: Efficient Action Tokenization for Vision-Language-Action ModelsKarl Pertsch, Kyle Stachowicz, Brian Ichter et al., 2025. The tokenizer compresses smooth trajectories better than independent uniform bins, but the decoded result is still meaningful only under the action convention used before tokenization.

Tokenization is therefore not an action space by itself. It is an encoding layered on joint, Cartesian or another continuous action space.

A selection rule

Choose the lowest-level action that the task requires and the platform can support safely.

Hold the controller, rate, horizon and observation history fixed when comparing policy architectures. Otherwise the experiment measures a package of interface changes. The action space is the package's most consequential part because it decides which errors learning must solve and which errors classical control is still allowed to reject.