Training robotic policies that reliably generalize to novel environments remains a persistent challenge. While state-of-the-art models leverage powerful global or dense visual features, these representations struggle to separate critical task-specific signals from background noise, causing failures under visual shifts.
In this work, we investigate Slot-based Object-Centric Representations (SOCRs) [1] as a structural solution that decomposes scenes into discrete, actionable entities without supervision. Through a large-scale diagnostic study across simulated (METAWORLD [4], LIBERO [5]) and real-world tasks, we systematically evaluate SOCRs against dominant baselines.
We find that structured abstraction drives inherent robustness: SOCRs drastically outperform standard models under severe lighting, texture, and clutter shifts without task-specific fine-tuning, and they scale effectively with in-domain video pre-training. However, we also identify a critical vulnerability: a structural capacity trade-off that leads to slot merging under high clutter. By mapping out both the strengths and the bottleneck limits of SOCRs, this work provides a clear roadmap for integrating structured visual abstractions into next-generation, generalizable robotic systems.
Our method extracts an object-centric representation from dense features using a Slot Attention mechanism [1]. Given an input image, it is first encoded by a DINOv2 visual backbone [2] into dense feature tokens. A Transformer layer handles temporal consistency before iteratively attending to features to extract a compact set of object representations (slots). We integrate this visual module, dubbed DINOSAUR* [3], into a multi-task policy training framework to evaluate manipulation generalization.
Figure 2: Overview of our robotic manipulation policy architecture. The pre-trained visual model extracts features that are combined with text instructions and proprioception in the observation trunk before passing to the action head.
To understand how these features aggregate information, we conducted a model focus analysis (Figure 1: Grad-CAM for CNNs, Attention-Rollout for ViTs). We found that while standard models tend to have broad or easily distracted focus areas, our slot-based representations split dense features into multiple slots that specialize in different objects. This explicitly separates task-relevant information (like the gripper and target object) from background noise.
While object-centric models excel at structured scene decomposition, they are primarily trained on in-the-wild datasets (like COCO). To bridge the domain gap for robotic manipulation, we introduced a domain-specific pretraining stage. Our DINOSAUR-Rob* model is pre-trained on a diverse, large-scale mixture of real-world robotic datasets including BridgeV2 [7], Fractal [8], and DROID [9]. We found that this diverse data mixture significantly outperforms models trained on single-source robotic datasets or standard vision baselines, particularly in real-world scenarios.
We evaluated DINOSAUR* and DINOSAUR-Rob* against dominant global (VC-1, R3M), dense (DINOv2, Theia, ResNet-50), and segmentation-driven (SAM+DINOv2) baselines. Our evaluation spanned simulated environments (MetaWorld, LIBERO) and real-world Franka deployments.
Figure 3: Overview of the real-world robotic setup using a Franka arm across four tabletop tasks: Stacking bowls, Screwdriver Drawer, Cans to bin, and Plates to stash.
Object-centric models generalize substantially better than dense and global representations under realistic out-of-distribution (OOD) scenarios.
Figure 4: Overview of generalization scenarios including novel distractors, unseen textures, and lighting changes applied to simulated environments (top) and real-world setups (bottom).
| Model | Distractors | Textures | Lighting | Overall OOD |
|---|---|---|---|---|
| ResNet-50 | 0.04 | 0.00 | 0.22 | 0.10 |
| DINOv2 | 0.11 | 0.03 | 0.39 | 0.18 |
| VC1 | 0.06 | 0.0 | 0.23 | 0.10 |
| Theia | 0.65 | 0.28 | 0.48 | 0.47 |
| DINOSAUR* (Ours) | 0.21 | 0.48 | 0.71 | 0.46 |
| DINOSAUR-Rob* (Ours) | 0.46 | 0.36 | 0.65 | 0.49 |
Table 1: Out-of-Distribution Success Rates in MetaWorld.
| Model | Distractors | Textures | Overall OOD |
|---|---|---|---|
| ResNet-50 | 0.15 | 0.10 | 0.12 |
| DINOv2 | 0.06 | 0.08 | 0.07 |
| VC1 | 0.03 | 0.02 | 0.03 |
| Theia | 0.06 | 0.08 | 0.07 |
| DINOSAUR* (Ours) | 0.27 | 0.29 | 0.28 |
| DINOSAUR-Rob* (Ours) | 0.37 | 0.44 | 0.41 |
Table 2: Out-of-Distribution Success Rates on the Real Robot.
Through qualitative analysis, we identified slot merging as the primary bottleneck for object-centric policies. When objects outnumber the available slots (K), features from novel distractors leak into task-relevant slots, polluting the state representation.
Figure 5: Qualitative analysis showing slot merging failures under hard, medium, and easy distractor levels (K=10).
Figure 6: The capacity trade-off: Impact of varying slot numbers (K) on In-Domain vs. Out-of-Domain success rates.
To rigorously test our policies under real-world conditions, we evaluated the Franka arm across a highly diverse set of configurations. The gallery below demonstrates the variance in object starting positions.
Figure 7: Real-world inference scene diversity showcasing task executions under varying spatial configurations and novel distractors.
@misc{chapin2026spotlightingtaskrelevantfeaturesobjectcentric,
title={Spotlighting Task-Relevant Features: Object-Centric Representations for Better Generalization in Robotic Manipulation},
author={Alexandre Chapin and Bruno Machado and Emmanuel Dellandréa and Liming Chen},
year={2026},
eprint={2601.21416},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2601.21416},
}