Unified Visuomotor Targets:
Supervising VLAs Beyond Physical Actions

Zhenyang Feng, Unnat Jain

University of California, Irvine

Accepted at IROS 2026

UVT replaces the raw action target with a single latent that encodes visual dynamics and robot actions together, improving real bimanual manipulation with no architectural changes and no additional data.

Abstract

Vision-Language-Action (VLA) models have become a practical framework for robot manipulation, leveraging pretrained vision-language models (VLMs) as multimodal backbones that map image observations and language instructions to robot actions. While recent work has improved action tokenization formats, the question of what information the prediction target should encode has received comparatively little attention. Joint positions and Cartesian poses are low-level, embodiment-specific signals that encode limited information about task context, creating a mismatch with the rich structure VLMs extract from visual and language inputs.

We propose UVT (Unified Visuomotor Target), a structured prediction target for VLA fine-tuning that integrates robot actions with discrete dynamics codes from a pretrained Latent Action Model (LAM). For each training sample, we construct a compact latent u_t ∈ ℜ³² that jointly encodes the action chunk and discrete dynamics code via a lightweight multimodal VAE. During VLA fine-tuning, the policy predicts u_t instead of raw actions, and a trainable decoder maps the predicted latent to executable action chunks. Because u_t encodes both motor control and visual transition dynamics, it is more consistent with the task structure that VLMs already extract from image and language inputs.

UVT requires no architectural changes, no additional data, and no modification to the training schedule. We evaluate on VLA-Adapter (regression head) and π_0.5 (diffusion/flow head) across LIBERO, LIBERO-Plus, and real bimanual manipulation tasks. UVT consistently improves convergence speed, final task performance, and policy robustness, with particularly strong gains under limited training budgets.

Method

Unified Visuomotor Target Construction

Unified Visuomotor Target u_t Training Architecture. A pretrained latent action model (LAM) extracts discrete dynamics codes z_t from image pairs, while ground-truth robot actions a_t ∈ ℜ^k×d provide continuous supervision. Two trainable encoders map a_t and z_t to Gaussian posteriors, which are fused via a weighted Product-of-Experts to produce the unified visuomotor target u_t ∈ ℜ³². The target is decoded to reconstruct both robot actions and discrete dynamics codes, while a KL divergence loss regularizes u_t toward a standard normal prior.

UVT for VLA Imitation Learning

After training the multimodal VAE, we precompute u_t for all training samples. During VLA fine-tuning, the policy predicts û_t from multimodal input and is supervised with a combined objective:

L_VLA = || û_t - u_t ||² + γ || â_t - a_t ||₁

This loss anchors the policy's internal representation to the unified visuomotor target space while ensuring decoded outputs remain consistent with executable joint commands. The modification changes only the prediction target and loss: the backbone VLA architecture remains unchanged, making UVT compatible with both regression-style and diffusion-based VLA policies.

Experiments

Real-World Bimanual Manipulation

Real-world evaluation platform. Two robot arms perform three tabletop tasks: Lift Pot, Close Marker, and Plate Handover, each testing different aspects of dual-arm coordination. For each task, we collect 30 demonstrations, fine-tune for 10 GPU hours, and run 50 evaluation trials.

Exemplar policy rollouts of UVT and task setup explanation.

Task	Method	# Succ	# Half	Rate
Lift Pot	VLA-Adapter	19	—	38.0%
Lift Pot	UVT (Ours)	27	—	54.0%
Close Marker	VLA-Adapter	0	0	0%
Close Marker	UVT (Ours)	5	28	38%
Plate Handover	VLA-Adapter	7	4	18%
Plate Handover	UVT (Ours)	12	16	40%

Over 50 trials per task, with Rate = (# Succ + 0.5 × # Half) / 50. UVT yields a 2.2x improvement on Plate Handover and reaches 38% on Close Marker, where the baseline fails every trial: UVT grasps both marker and cap consistently, and completes the precision insertion on 5 trials. Lift Pot is scored on full success only.

UVT Rollouts

Uncut policy rollouts on the physical bimanual platform. Select a task to view its rollouts.

Task Selection

UVT Rollout 1

UVT Rollout 2

UVT Rollout 3

UVT Rollout 1

UVT Rollout 2

UVT Rollout 3

UVT Rollout 1

Simulation Benchmarks

UVT policy rollouts on LIBERO. One successful rollout per suite, rendered offline in Blender Cycles from the recorded simulator state. Below each rollout we plot the x, y, and z components of the action chunk the policy actually executed at that timestep. The policy produces smooth, task-coherent motions while completing diverse manipulation objectives.

LIBERO benchmark suites. The four suites (Object, Spatial, Goal, and Long) each include distinct manipulation challenges. LIBERO-Plus provides additional environmental perturbations to test model robustness.

Training Efficiency

We compare success rates at 10k and 100k training steps across LIBERO and LIBERO-Plus. UVT achieves significantly faster performance gains under limited computation, especially on the more challenging LIBERO-Plus benchmark.

Environment	Method	Object		Spatial		Goal		Long		Avg.
Environment	Method	10k	100k	10k	100k	10k	100k	10k	100k	10k	100k
LIBERO	VLA-Adapter	93.4	97.6	78.8	96.2	90.4	95.6	56.2	89.0	79.7	94.6
LIBERO	UVT (Ours)	94.0	99.6	95.0	98.6	95.0	97.2	68.8	91.2	88.2	96.7
LIBERO-Plus	VLA-Adapter	52.7	58.3	34.0	86.8	42.3	72.4	41.7	64.3	42.7	70.5
LIBERO-Plus	UVT (Ours)	52.4	60.4	81.5	87.2	67.8	74.7	63.0	72.1	66.2	73.6

At 10k steps on LIBERO-Plus Spatial, UVT achieves 81.5% vs. baseline's 34.0%, a +47.5% improvement under identical training budgets.

Benchmark Results (100k Steps)

UVT consistently improves performance over baselines across different VLA architectures without increasing model size, demonstrating compatibility with both regression-style (VLA-Adapter) and diffusion-based (π_0.5) policies.

Model	Object	Spatial	Goal	Long	Avg.
VLA-Adapter	97.6	96.2	95.6	89.0	94.6
π_0.5	97.8	98.4	98.0	90.6	96.2
UVT (VLA-Adapter)	99.6	98.6	97.2	91.2	96.7
UVT (π_0.5)	99.0	98.4	97.8	92.4	96.9

Analysis

Ablation Study

We compare alternative ways of incorporating discrete dynamics codes into VLA training. Simply adding an auxiliary dynamics prediction head provides moderate gains, but fusing both signals into a unified target via UVT yields substantially stronger improvements.

Method	@ 10k steps	@ 100k steps
VLA-Adapter	78.8	96.2
w/ Villa-X head	83.2	96.8
w/ UniVLA head	86.4	95.8
UVT (Ours)	95.0	98.6

Ablation on LIBERO-Spatial. Independent heads supervise separate outputs but do not enforce interaction between motor and dynamics representations. UVT encodes both jointly.

Latent Representation Analysis

t-SNE visualization of unified visuomotor target u_t and raw robot action a_t. Compared to raw robot actions, the unified visuomotor target exhibits more structured and smoothly organized embeddings across LIBERO suites. Even without suite information during training, the latent naturally groups actions from the same suite together, suggesting a more semantically consistent representation of task dynamics.

Decoded Action Smoothness

Temporal profiles of decoded and raw action trajectories. Robot actions decoded from sampled u_t are noticeably smoother, exhibiting fewer high-frequency fluctuations while preserving the overall motion trend. This suggests that the learned u_t captures underlying motion dynamics while filtering out low-level actuator noise. Pick an action dimension and drag the slider (or press play) to step through the chunk; the three chunks shown are the roughest in LIBERO-Long by total variation and direction-change count, and both panels share a y-axis for direct comparison.

BibTeX

@misc{feng2026unifiedvisuomotortargetssupervising,
      title={Unified Visuomotor Targets: Supervising VLAs Beyond Physical Actions}, 
      author={Zhenyang Feng and Unnat Jain},
      year={2026},
      eprint={2608.03563},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2608.03563}, 
}