MRPose: Multi-Robot Relative 6D Pose Estimation for Unseen Objects From RGB Images

Jiale Ren , Mingxin Tan , Mengyuan Liu , Yingxuan You , Hong Liu

CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (5) : 1430 -1443.

PDF (3854KB)
CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (5) :1430 -1443. DOI: 10.1049/cit2.70176
ORIGINAL RESEARCH
research-article
MRPose: Multi-Robot Relative 6D Pose Estimation for Unseen Objects From RGB Images
Author information +
History +
PDF (3854KB)

Abstract

Relative pose estimation under the query–reference paradigm has emerged as a practical alternative for estimating the 6D pose of unseen objects without relying on CAD models or extensive annotations. However, many existing approaches remain difficult to deploy in practice, as their reliance on geometric matching makes robustness highly sensitive to matching quality. In this paper, we present MRPose, a framework for relative 6D object pose estimation under a multi-robot setting, where query and reference views are independently captured by different robots. Without requiring CAD models, shared calibration across robots or pose annotations, MRPose directly estimates the full relative transformation between unseen objects from RGB images. Built upon the power of vision foundation models, MRPose introduces a multiplex feature learning strategy that jointly exploits patch-level appearance correspondence and sparse geometric cues through attention-based cross-view feature aggregation. Instead of relying on traditional geometric solvers such as PnP or essential matrix estimation, the proposed framework directly regresses the full relative 6D pose in an end-to-end manner. Extensive experiments on the LINEMOD and YCB-Video benchmarks demonstrate that MRPose compares favourably with existing RGB-based relative pose estimation methods. Real-world experiments further validate its applicability in practical scenarios.

Keywords

6D object pose estimation / cross-view feature interaction / relative pose estimation

Cite this article

Download citation ▾
Jiale Ren, Mingxin Tan, Mengyuan Liu, Yingxuan You, Hong Liu. MRPose: Multi-Robot Relative 6D Pose Estimation for Unseen Objects From RGB Images. CAAI Transactions on Intelligence Technology, 2026, 11 (5) : 1430-1443 DOI:10.1049/cit2.70176

登录浏览全文

4963

注册一个新账户 忘记密码

References

[1]

S. Wang, M. N. Nikolić, T. L. Lam, Q. Gao, R. Ding, and T. Zhang, “ Robot Manipulation Based on Embodied Visual Perception: A Survey,” CAAI Transactions on Intelligence Technology 10, no. 4 (2025): 945-958, https://doi.org/10.1049/cit2.70022.

[2]

Y. Chen, S. Huang, T. Yuan, S. Qi, Y. Zhu, and S.-C. Zhu, “ Holistic++ Scene Understanding: Single-View 3d Holistic Scene Parsing and Human Pose Estimation With Human-Object Interaction and Physical Commonsense,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2019), 8648-8657.

[3]

Y. Zhao, S. Zhang, W. Zhao, Y. Wei, and J. Peng, “ Augmented Reality System Based on Real-Time Object 6d Pose Estimation,” in Proceedings of the 2023 2nd International Conference on Image Processing and Media Computing (ICIPMC) (IEEE, 2023), 27-34.

[4]

Y. Xiang, T. Schmidt, V. Narayanan, and D. Fox, “ Posecnn: A Convolutional Neural Network for 6d Object Pose Estimation in Cluttered Scenes,” preprint arXiv:1711.00199 (2017).

[5]

W. Kehl, F. Manhardt, F. Tombari, S. Ilic, and N. Navab, “ Ssd-6d: Making Rgb-Based 3d Detection and 6d Pose Estimation Great Again,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017), 1521-1529.

[6]

C. Song, J. Song, and Q. Huang, “ Hybridpose: 6D Object Pose Estimation Under Hybrid Representations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020), 431-440.

[7]

Y. Bukschat and M. Vetter, “ Efficientpose: An Efficient, Accurate and Scalable End-to-End 6d Multi Object Pose Estimation Approach,” preprint arXiv:2011.04307 (2020).

[8]

H. Wang, S. Sridhar, J. Huang, J. Valentin, S. Song, and L. J. Guibas, “ Normalized Object Coordinate Space for Category-Level 6d Object Pose and Size Estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), 2642-2651.

[9]

K. Chen and Q. Dou, “ Sgpa: Structure-Guided Prior Adaptation for Category-Level 6d Object Pose Estimation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2021), 2773-2782.

[10]

D. Chen, J. Li, Z. Wang, and K. Xu, “ Learning Canonical Shape Space for Category-Level 6d Object Pose and Size Estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020), 11973-11982.

[11]

J. Lin, Z. Wei, Z. Li, S. Xu, K. Jia, and Y. Li, “ Dualposenet: Category-Level 6d Object Pose and Size Estimation Using Dual Pose Network With Refined Learning of Pose Consistency,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR) (2021), 3560-3569.

[12]

Z. Fan, Z. Song, Z. Wang, et al., “ Acr-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6d Object Pose Estimation,” in Proceedings of the 2024 International Conference on Multimedia Retrieval (ICMR) (2024), 55-63.

[13]

T. Hou, A. Ahmadyan, L. Zhang, J. Wei, and M. Grundmann, “ Mobilepose: Real-Time Pose Estimation for Unseen Objects With Weak Shape Supervision,” preprint arXiv:2003.03522 (2020).

[14]

M. Tian, M. H. Ang, and G. H. Lee, “ Shape Prior Deformation for Categorical 6d Object Pose and Size Estimation,” in Proceedings of the European Conference on Computer Vision (ECCV) (2020), 530-546.

[15]

A. S. Periyasamy, A. Amini, V. Tsaturyan, and S. Behnke, “ Yolopose v2: Understanding and Improving Transformer-Based 6d Pose Estimation,” Robotics and Autonomous Systems 168 (2023): 104490, https://doi.org/10.1016/j.robot.2023.104490.

[16]

Z. Li, G. Wang, and X. Ji, “ Cdpn: Coordinates-Based Disentangled Pose Network for Real-Time Rgb-Based 6-dof Object Pose Estimation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2019), 7678-7687.

[17]

A. Amini, A. S. Periyasamy, and S. Behnke, “ T6d-Direct: Transformers for Multi-Object 6d Pose Direct Regression,” in Proceedings of the DAGM German Conference on Pattern Recognition (GCPR) (Springer, 2021), 530-544.

[18]

M. Zheng, P. Gao, R. Zhang, et al., “ End-to-End Object Detection With Adaptive Clustering Transformer,” preprint arXiv:2011.09315 (2020).

[19]

A. Vaswani, N. Shazeer, N. Parmar, et al., “ Attention Is All You Need,” in Neural Information Processing Systems (NIPS) (2017), 5998-6008.

[20]

J. Chen, Z. Zhou, M. Sun, et al., “ Zeropose: Cad-Prompted Zero-Shot Object 6d Pose Estimation in Cluttered Scenes,” IEEE Transactions on Circuits and Systems for Video Technology 35, no. 2 (2024): 1251-1264, https://doi.org/10.1109/TCSVT.2024.3482439.

[21]

J. Lin, L. Liu, D. Lu, and K. Jia, “ Sam-6d: Segment Anything Model Meets Zero-Shot 6d Object Pose Estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024), 27906-27916.

[22]

Y. He, Y. Wang, H. Fan, J. Sun, and Q. Chen, “ Fs6d: Few-Shot 6d Pose Estimation of Novel Objects,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), 6814-6824.

[23]

B. Wen, W. Yang, J. Kautz, and S. Birchfield, “ Foundationpose: Unified 6d Pose Estimation and Tracking of Novel Objects,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024), 17868-17879.

[24]

T. Nguyen, M. N. Bui, C. Nguyen, et al., “ Nope: Novel Object Pose Estimation From a Single Image,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024).

[25]

X. Liu, G. Wang, R. Zhang, C. Zhang, F. Tombari, and X. Ji, “ Unopose: Unseen Object Pose Estimation With an Unposed rgb-d Reference Image,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025), 22023-22034.

[26]

T. Lee, B. Wen, M. Kang, G. Kang, I. S. Kweon, and K.-J. Yoon, “ Any6d: Model-Free 6d Pose Estimation of Novel Objects,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025), 11633-11643.

[27]

V. Leroy, Y. Cabon, and J. Revaud, “ Grounding Image Matching in 3d With mast3r,” in Proceedings of the European Conference on Computer Vision (ECCV) (2024), 71-91.

[28]

W. Deng, D. Campbell, C. Sun, et al., “ Pos3r: 6D Pose Estimation for Unseen Objects Made Easy,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025), 16818-16828.

[29]

J. Sun, Z. Wang, S. Zhang, et al., “ Onepose: One-Shot Object Pose Estimation Without Cad Models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), 6825-6834.

[30]

X. He, J. Sun, Y. Wang, D. Huang, H. Bao, and X. Zhou, “ Onepose++: Keypoint-Free One-Shot Object Pose Estimation Without Cad Models,” Advances in Neural Information Processing Systems 35 (2022): 35103-35115, https://doi.org/10.52202/068431-2544.

[31]

Y. Liu, Y. Wen, S. Peng, et al., “ Gen6d: Generalizable Model-Free 6-Dof Object Pose Estimation From Rgb Images,” in Proceedings of the European Conference on Computer Vision (ECCV) (2022), 298-315.

[32]

Z. Fan, P. Pan, P. Wang, Y. Jiang, D. Xu, and Z. Wang, “ Pope: 6-Dof Promptable Pose Estimation of Any Object in Any Scene With One Reference,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024), 7771-7781.

[33]

V. N. Nguyen, T. Groueix, G. Ponimatkin, V. Lepetit, and T. Hodan, “ Cnos: A Strong Baseline for Cad-Based Novel Object Segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops (2023), 2134-2140.

[34]

Y. Xiong, B. Varadarajan, L. Wu, et al., “ Efficientsam: Leveraged Masked Image Pretraining for Efficient Segment Anything,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024), 16111-16121.

[35]

M. Oquab, T. Darcet, T. Moutakanni, et al., “ Dinov2: Learning Robust Visual Features Without Supervision,” preprint arXiv:2304.07193 (2023).

[36]

Y. Zhou, C. Barnes, J. Lu, J. Yang, and H. Li, “ On the Continuity of Rotation Representations in Neural Networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), 5745-5753.

[37]

V. N. Nguyen, Y. Hu, Y. Xiao, M. Salzmann, and V. Lepetit, “ Templates for 3d Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions,” in Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2022), 6771-6780.

[38]

S. Hinterstoisser, V. Lepetit, S. Ilic, et al., “ Model Based Training, Detection and Pose Estimation of Texture-Less 3d Objects in Heavily Cluttered Scenes,” in Proceedings of the Asian Conference on Computer Vision (ACCV) (2013), 548-562.

[39]

B. Calli, A. Singh, A. Walsman, S. Srinivasa, P. Abbeel, and A. M. Dollar, “ The Ycb Object and Model Set: Towards Common Benchmarks for Manipulation Research,” in Proceedings of the International Conference on Advanced Robotics (ICAR) (2015), 510-517.

[40]

W. Goodwin, S. Vaze, I. Havoutis, and I. Posner, “ Zero-Shot Category-Level Object Pose Estimation,” in Proceedings of the European Conference on Computer Vision (ECCV) (2022), 516-532.

[41]

J. Sun, Z. Shen, Y. Wang, H. Bao, and X. Zhou, “ Loftr: Detector-Free Local Feature Matching With Transformers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 8922-8931.

[42]

J. Yang, Y. Gao, D. Li, and S. L. Waslander, “ Robi: A Multi-View Dataset for Reflective Objects in Robotic Bin-Picking,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2021), 9788-9795.

[43]

J. Huang, J. Liang, J. Hu, et al., “ Xyz-Ibd: A High-Precision Bin-Picking Dataset for Object 6d Pose Estimation Capturing Real-World Industrial Complexity,” preprint arXiv:2506.00599 (2025).

PDF (3854KB)

0

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉