Experiments
This section introduces experiments conducted to investigate the research questions, together with evaluation datasets, metrics, and implementation details.
Environmental and Social Context Disentanglement
To evaluate the impact of the proposed strategy under domain-incremental learning, the Explicit Disentanglement Dual-Branch Network (EDD) was compared against three types of state-of-the-art baselines:
- A robotics-specific continual-learning method for social appropriateness.
- Recent domain-incremental computer-vision methods.
- Pretrained vision–language models.
Baseline model details and their rationale are reported in the corresponding comparison table.
Overall Appropriateness Prediction
The overall appropriateness prediction performance of EDD was compared with baseline methods. EDD achieves the lowest root mean square error (RMSE) and the highest Pearson correlation coefficient (PCC) and concordance correlation coefficient (CCC), indicating more accurate and consistent action-appropriateness predictions than the compared baselines.
FedLGR yields the highest error and near-zero PCC and CCC, indicating the weakest overall performance among the compared models. DARE++ achieves the strongest baseline correlations, followed by DUCA. Vision–language model methods, including Qwen2.5-3B-Instruct, LLaVA-OV-0.5B, and DeepSeek-1.3B-Chat, yield lower PCC and CCC values.
Continual-Learning Ablations
EDD ablations were compared across different training regimes. Dual-branch continual learning, which represents the proposed approach, attains lower error, higher PCC and CCC, and lower forgetting, measured by backward transfer (BWT), than the single-branch ablation that does not use image decomposition.
Fine-tuning without replay degrades performance for both architectures. It increases RMSE, lowers PCC and CCC, and produces higher BWT.
Joint-training variants, which have access to all domains simultaneously, act as an empirical upper bound and achieve the best overall accuracy. The dual-branch continual-learning model is closer to this upper bound than the single-branch continual-learning model.
Overall, compared with the baselines and the non-disentangled variant, EDD shows consistently favourable performance across all metrics, better alignment with human-provided appropriateness scores, and lower forgetting. These results suggest improved domain-incremental learning of socially appropriate robot actions.
Impact of Disentanglement Strategies
Alternative image decomposition strategies were compared for the dual-branch network. Across all splits, the bounding-box strategy yields the lowest RMSE and highest PCC and CCC.
Robot close-up and no decomposition obtain similar RMSE values, both lower than the silhouette method. No decomposition shows consistently higher correlations than the robot close-up and silhouette methods.
The forgetting measure, BWT, remains low across all variants. The bounding-box and robot close-up strategies achieve the lowest BWT, indicating lower forgetting across domains than the silhouette and no-decomposition strategies.
Overall, all methods perform relatively similarly, with the bounding-box strategy achieving the lowest prediction error and strongest correlations.
Discussion, Conclusion, and Future Work
This work highlights a key insight for continual learning in socially appropriate robotics: making environmental and social cues explicit can support learning action appropriateness under domain shift.
In this setting, these cues are addressed through the Explicit Disentanglement Dual-Branch framework. The framework separates the cues at the input level and fuses them with a dual-branch architecture trained using replay-based rehearsal.
The results show that the EDD framework outperforms several state-of-the-art baselines and generates appropriateness scores that more closely align with human-provided scores across complementary metrics. This explicit separation also enables targeted analysis of how disentanglement strategies and domain order shape continual-learning performance for socially appropriate actions.
Several directions remain for future research. EDD can be extended beyond the current single-robot embodiment, and its robustness can be examined under embodiment shifts.
Future work can further examine EDD in real-world settings to assess its generalisability beyond synthetic scenes. This includes robustness to synthetic-to-real shifts and the effects of imperfect panoptic segmentation on environmental–social decomposition.
EDD’s structure also invites work on explainability. Future research can analyse what each branch learns and how predictions depend on environmental versus social cues.
Another promising direction is adaptive fusion, in which the model dynamically adjusts to the separate branches based on task context, such as the robot action, and the interaction environment, such as a home or office.
Frequently Asked Questions
What is the Explicit Disentanglement Dual-Branch Network? EDD is a dual-branch framework that separates environmental and social cues at the input level and fuses them for learning socially appropriate robot actions.
How does EDD perform compared with the evaluated baselines? EDD achieves the lowest RMSE and highest PCC and CCC, indicating more accurate and consistent appropriateness predictions.
Which image decomposition strategy performs best? The bounding-box strategy achieves the lowest prediction error and strongest correlations across the evaluated splits.
What are the main directions for future work? Future work includes embodiment shifts, real-world evaluation, synthetic-to-real robustness, explainability, and adaptive fusion based on task and interaction context.