Vision-Language-Action (VLA) models have become a practical framework for robot manipulation, leveraging pretrained vision-language models (VLMs) as multimodal backbones that map image observations and language instructions to robot actions. While recent work has improved action tokenization formats, the question of what information the prediction target should encode has received comparatively little attention. Joint positions and Cartesian poses are low-level, embodiment-specific signals that encode limited information about task context, creating a mismatch with the rich structure VLMs extract from visual and language inputs.
We propose UVT (Unified Visuomotor Target), a structured prediction target for VLA fine-tuning that integrates robot actions with discrete dynamics codes from a pretrained Latent Action Model (LAM). For each training sample, we construct a compact latent ut ∈ ℜ32 that jointly encodes the action chunk and discrete dynamics code via a lightweight multimodal VAE. During VLA fine-tuning, the policy predicts ut instead of raw actions, and a trainable decoder maps the predicted latent to executable action chunks. Because ut encodes both motor control and visual transition dynamics, it is more consistent with the task structure that VLMs already extract from image and language inputs.
UVT requires no architectural changes, no additional data, and no modification to the training schedule. We evaluate on VLA-Adapter (regression head) and π0.5 (diffusion/flow head) across LIBERO, LIBERO-Plus, and real bimanual manipulation tasks. UVT consistently improves convergence speed, final task performance, and policy robustness, with particularly strong gains under limited training budgets.
Unified Visuomotor Target ut Training Architecture. A pretrained latent action model (LAM) extracts discrete dynamics codes zt from image pairs, while ground-truth robot actions at ∈ ℜk×d provide continuous supervision. Two trainable encoders map at and zt to Gaussian posteriors, which are fused via a weighted Product-of-Experts to produce the unified visuomotor target ut ∈ ℜ32. The target is decoded to reconstruct both robot actions and discrete dynamics codes, while a KL divergence loss regularizes ut toward a standard normal prior.
After training the multimodal VAE, we precompute ut for all training samples. During VLA fine-tuning, the policy predicts ût from multimodal input and is supervised with a combined objective:
LVLA = || ût - ut ||2 + γ || ât - at ||1
This loss anchors the policy's internal representation to the unified visuomotor target space while ensuring decoded outputs remain consistent with executable joint commands. The modification changes only the prediction target and loss — the backbone VLA architecture remains unchanged, making UVT compatible with both regression-style and diffusion-based VLA policies.
Real-world evaluation platform. Two robot arms perform three tabletop tasks: Lift Pot, Close Marker, and Plate Handover, each testing different aspects of dual-arm coordination. For each task, we collect 30 demonstrations, fine-tune for 10 GPU hours, and run 50 evaluation trials.
Real-world experiment video overview:
| Task | Method | # Succ | # Half | Rate |
|---|---|---|---|---|
| Lift Pot | VLA-Adapter | 19 | — | 38.0% |
| UVT (Ours) | 27 | — | 54.0% | |
| Close Marker | VLA-Adapter | 0 | 0 | 0% |
| UVT (Ours) | 0 | 26 | 26% | |
| Plate Handover | VLA-Adapter | 7 | 4 | 18% |
| UVT (Ours) | 12 | 16 | 40% |
UVT yields a 2.2x improvement on Plate Handover and enables marker insertion on Close Marker where the baseline achieves 0% success.
LIBERO benchmark suites. The four suites — Object, Spatial, Goal, and Long — each include distinct manipulation challenges. LIBERO-Plus provides additional environmental perturbations to test model robustness.
We compare success rates at 10k and 100k training steps across LIBERO and LIBERO-Plus. UVT achieves significantly faster performance gains under limited computation, especially on the more challenging LIBERO-Plus benchmark.
| Environment | Method | Object | Spatial | Goal | Long | Avg. | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 10k | 100k | 10k | 100k | 10k | 100k | 10k | 100k | 10k | 100k | ||
| LIBERO | VLA-Adapter | 93.4 | 97.6 | 78.8 | 96.2 | 90.4 | 95.6 | 56.2 | 89.0 | 79.7 | 94.6 |
| UVT (Ours) | 94.0 | 99.6 | 95.0 | 98.6 | 95.0 | 97.2 | 68.8 | 91.2 | 88.2 | 96.7 | |
| LIBERO-Plus | VLA-Adapter | 52.7 | 58.3 | 34.0 | 86.8 | 42.3 | 72.4 | 41.7 | 64.3 | 42.7 | 70.5 |
| UVT (Ours) | 52.4 | 60.4 | 81.5 | 87.2 | 67.8 | 74.7 | 63.0 | 72.1 | 66.2 | 73.6 | |
At 10k steps on LIBERO-Plus Spatial, UVT achieves 81.5% vs. baseline's 34.0% — a +47.5% improvement under identical training budgets.
UVT consistently improves performance over baselines across different VLA architectures without increasing model size, demonstrating compatibility with both regression-style (VLA-Adapter) and diffusion-based (π0.5) policies.
| Model | Object | Spatial | Goal | Long | Avg. |
|---|---|---|---|---|---|
| VLA-Adapter | 97.6 | 96.2 | 95.6 | 89.0 | 94.6 |
| π0.5 | 97.8 | 98.4 | 98.0 | 90.6 | 96.2 |
| UVT (VLA-Adapter) | 99.6 | 98.6 | 97.2 | 91.2 | 96.7 |
| UVT (π0.5) | 99.0 | 98.4 | 97.8 | 92.4 | 96.9 |
Example UVT policy rollouts across LIBERO tasks, each shown alongside its corresponding decoded action trajectory. The policy produces smooth, task-coherent motions while successfully completing diverse manipulation objectives.
We compare alternative ways of incorporating discrete dynamics codes into VLA training. Simply adding an auxiliary dynamics prediction head provides moderate gains, but fusing both signals into a unified target via UVT yields substantially stronger improvements.
| Method | @ 10k steps | @ 100k steps |
|---|---|---|
| VLA-Adapter | 78.8 | 96.2 |
| w/ Villa-X head | 83.2 | 96.8 |
| w/ UniVLA head | 86.4 | 95.8 |
| UVT (Ours) | 95.0 | 98.6 |
Ablation on LIBERO-Spatial. Independent heads supervise separate outputs but do not enforce interaction between motor and dynamics representations. UVT encodes both jointly.
t-SNE visualization of unified visuomotor target ut and raw robot action at. Compared to raw robot actions, the unified visuomotor target exhibits more structured and smoothly organized embeddings across LIBERO suites. Even without suite information during training, the latent naturally groups actions from the same suite together, suggesting a more semantically consistent representation of task dynamics.
Temporal profiles of decoded and raw action trajectories. Robot actions decoded from sampled ut are noticeably smoother, exhibiting fewer high-frequency fluctuations while preserving the overall motion trend. This suggests that the learned ut captures underlying motion dynamics while filtering out low-level actuator noise.
Coming soon.