SVLGaussian: Single View Language Gaussian Splatting

Abstract

Open-vocabulary 3D querying based on 3D Gaussian splatting (3DGS) shows great promise in facilitating accurate 3D query capabilities of AI systems. These methods typically rely on pre-captured multi-view images to enable natural language interactions with 3D scenes. In practice, when embodied AI encounters unexplored scenes, it is difficult to obtain observations from different viewpoints beforehand. This challenge highlights the importance of exploring natural language-driven 3D scene querying from a single current viewpoint. This paper proposes single view language Gaussian splatting (SVLGaussian) for the novel task: Open-vocabulary 3D querying based on the input single view. By leveraging multi-round inference of multimodal large language models, SVLGaussian efficiently generates pixel-level semantic probabilities and rapidly embeds them into a 3D Gaussian field, enabling real-time language-guided semantic querying. To verify our model, we annotated three datasets: Lerf_ovs and 3D-OVS, which are tailored for open-vocabulary 3D querying, and RE10K, which is adapted for single-view 3D reconstruction. Both quantitative and qualitative results show that our method effectively supports open-vocabulary 3D querying from a single view.

Keywords

Gaussian splatting / language-driven semantic querying / single-view 3D reconstruction

Acknowledgements

This work was supported by the Key Research and Development Program of Zhejiang Province (Grant No. 2025C01026); the National College Students Innovation and Entrepreneurship Training Program of the Ministry of Education of the People's Republic of China (Grant No. 202410336024); the National Natural Science Foundation of China (Grant Nos. 62206082, 62402152, 62472133 and 62422204) and the Natural Science Foundation of Zhejiang Province (Grant Nos. LDT23F02025F02, LQN25F020017 and LRG26F020001).

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

[1] M. Ji, R.-Z. Qiu, X. Zou, and X. Wang, “Graspsplats: Efficient Manipulation With 3D Feature Splatting,” preprint, arXiv:2409.02084 (2024).

[2] A. Rashid, S. Sharma, C. M. Kim, et al., Language Embedded Radiance Fields for Zero-Shot Task-Oriented Grasping,” in 7th Annual Conference on Robot Learning (2023), https://openreview.net/forum?id=k-Fg8JDQmc.

[3] M. Qin, W. Li, J. Zhou, H. Wang, and H. Pfister, “Langsplat: 3D Language Gaussian Splatting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), 20051-20060.

[4] S. Szymanowicz, E. Insafutdinov, C. Zheng, et al., Flash3d: Feed-Forward Generalisable 3D Scene Reconstruction From a Single Image,” preprint, arXiv:2406.04343 (2024).

[5] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, “3D Gaussian Splatting for Real-Time Radiance Field Rendering,” ACM Transactions on Graphics 42, no. 4 (2023): 139-141, https://doi.org/10.48550/arXiv.2308.04079.

[6] J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman, “Mip-Nerf 360: Unbounded Anti-Aliased Neural Radiance Fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), 5470-5479.

[7] Q. Cao, J. Ding, Q. Tang, et al., GC-GS: Gradient Control Gaussian Splatting With Various Image Degradation,” Pattern Recognition 171 (2026): 112304, https://doi.org/10.1016/J.PATCOG.2025.112304.

[8] C. Peng, Y. Tang, Y. Zhou, et al., Bags: Blur Agnostic Gaussian Splatting Through Multi-Scale Kernel Modeling,” in European Conference on Computer Vision (Springer, 2024), 293-310.

[9] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing Scenes as Neural Radiance Fields for View Synthesis,” Communications of the ACM 65, no. 1 (2021): 99-106, https://doi.org/10.1145/3503250.

[10] Z. Yu, A. Chen, B. Huang, T. Sattler, and A. Geiger, “Mip-Splatting: Alias-Free 3D Gaussian Splatting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), 19447-19456.

[11] A. Chen, Z. Xu, F. Zhao, et al., Mvsnerf: Fast Generalizable Radiance Field Reconstruction From Multi-View Stereo,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2021), 14124-14133.

[12] Y. Chen, H. Xu, C. Zheng, et al., Mvsplat: Efficient 3D Gaussian Splatting From Sparse Multi-View Images,” in European Conference on Computer Vision (Springer, 2024), 370-386.

[13] X. Long, C. Lin, P. Wang, T. Komura, and W. Wang, “Sparseneus: Fast Generalizable Neural Surface Reconstruction From Sparse Views,” in European Conference on Computer Vision (Springer, 2022), 210-227.

[14] Y. Chen, H. Xu, Q. Wu, C. Zheng, T.-J. Cham, and J. Cai, “Explicit Correspondence Matching for Generalizable Neural Radiance Fields,” preprint, arXiv:2304.12294 (2023).

[15] M. Ding, Z. Wang, J. Wang, et al., Futuregs: Structured Gaussian Fields for Future-Aware Dynamic Scene Modeling,” in Proceedings of the 33rd ACM International Conference on Multimedia (2025), 8322-8331.

[16] X. Liu, C. Zhou, and S. Huang, “3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting With View-Consistent 2D Diffusion Priors,” Advances in Neural Information Processing Systems 37 (2024): 133305-133327, https://doi.org/10.48550/arXiv.2410.16266.

[17] R. Wu, B. Mildenhall, P. Henzler, et al., Reconfusion: 3D Reconstruction With Diffusion Priors,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), 21551-21561.

[18] D. Charatan, S. L. Li, A. Tagliasacchi, and V. Sitzmann, “Pixelsplat: 3D Gaussian Splats From Image Pairs for Scalable Generalizable 3D Reconstruction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), 19457-19467.

[19] A. Hanson, A. Tu, G. Lin, V. Singla, M. Zwicker, and T. Goldstein, “Speedy-Splat: Fast 3D Gaussian Splatting With Sparse Pixels and Sparse Primitives,” preprint, arXiv:2412.00578 (2024).

[20] J. Li, J. Zhang, X. Bai, et al., Dngaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields With Global-Local Depth Normalization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), 20775-20785.

[21] J. Wang, L. Zhang, Y. Ding, et al., Gf-Gssr: Gradient Fused Gaussian Splatting Super-Resolution,” in 2025 IEEE 2nd International Conference on Deep Learning and Computer Vision (DLCV) (2025), 1-6.

[22] J. Zhang, J. Li, X. Yu, et al., Cor-Gs: Sparse-view 3D Gaussian Splatting via co-regularization,” in European Conference on Computer Vision (Springer, 2024), 335-352.

[23] C. Godard, O. Mac Aodha, and G. J. Brostow, “Unsupervised Monocular Depth Estimation With Left-Right Consistency,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017), 270-279.

[24] I. Laina, C. Rupprecht, V. Belagiannis, F. Tombari, and N. Navab, “Deeper Depth Prediction With Fully Convolutional Residual Networks,” in 2016 Fourth International Conference on 3D Vision (3DV) (IEEE, 2016), 239-248.

[25] J. Li, Z. Feng, Q. She, H. Ding, C. Wang, and G. H. Lee, “MINE: Towards Continuous Depth MPI With NeRF for Novel View Synthesis,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2021), 12578-12588.

[26] R. Tucker and N. Snavely, “Single-View View Synthesis With Multiplane Images,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020), 551-560.

[27] S. Tulsiani, R. Tucker, and N. Snavely , Layer-Structured 3D Scene Inference via View Synthesis (2018), 311-327, https://doi.org/10.1007/978-3-030-01234-2_19.

[28] Z. Li, T. Dekel, F. Cole, et al., Learning the Depths of Moving People by Watching Frozen People,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019), 4521-4530.

[29] R. Ranftl, A. Bochkovskiy, and V. Koltun, “Vision Transformers for Dense Prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2021), 12179-12188.

[30] S. Shao, Z. Pei, W. Chen, X. Wu, and Z. Li , Nddepth: Normal-Distance Assisted Monocular Depth Estimation (2023).

[31] J. Chibane, A. Bansal, V. Lazova, and G. Pons-Moll, “Stereo Radiance Fields (Srf): Learning View Synthesis for Sparse Views of Novel Scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021), 7911-7920.

[32] P. Henzler, J. Reizenstein, P. Labatut, et al., Unsupervised Learning of 3D Object Categories From Videos in the Wild,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021), 4700-4709.

[33] S. Szymanowicz, C. Rupprecht, and A. Vedaldi , Splatter Image: Ultra-Fast Single-View 3D Reconstruction (2023).

[34] S. Kobayashi, E. Matsumoto, and V. Sitzmann , Decomposing Nerf for Editing via Feature Field Distillation (2022).

[35] K. Liu, F. Zhan, J. Zhang, et al., Weakly Supervised 3D Open-Vocabulary Segmentation,” Advances in Neural Information Processing Systems 36 (2023): 53433-53456, https://doi.org/10.48550/arXiv.2305.14093.

[36] Y. Siddiqui, L. Porzi, S. R. Buló, et al., Panoptic Lifting for 3D Scene Understanding With Neural Fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), 9043-9052.

[37] S. Zhi, T. Laidlow, S. Leutenegger, and A. J. Davison, “In-place Scene Labelling and Understanding With Implicit Scene Representation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2021), 15838-15847.

[38] J. Ding, Z. Wang, Y. Liu, et al., Fuzzy Language Gaussian Splatting,” IEEE Transactions on Fuzzy Systems 34, no. 4 (2026): 1165-1174, https://doi.org/10.1109/TFUZZ.2025.3644901.

[39] J. Kerr, C. M. Kim, K. Goldberg, A. Kanazawa, and M. Tancik, “Lerf: Language Embedded Radiance Fields,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), 19729-19739.

[40] M. Zhu, Y. Gong, D. Gu, and C. Tian, “Improving 3D Object Detection in Neural Radiance Fields With Channel Attention,” CAAI Transactions on Intelligence Technology 10, no. 5 (2025): 1446-1458, https://doi.org/10.1049/cit2.70045.

[41] K. Chen, B. Dai, M. Qin, et al., Slgaussian: Fast Language Gaussian Splatting in Sparse Views,” in Proceedings of the 33rd ACM International Conference on Multimedia (Association for Computing Machinery, 2025), 3047-3056, https://doi.org/10.1145/3746027.3754964.

[42] J. Hu, Z. Chen, Z. Li, Y. Xu, and J. Zhang, “Sparselgs: Sparse View Language Embedded Gaussian Splatting,” preprint, arXiv:2412.02245 (2024).

[43] X. Wang, C. Lan, H. Zhu, Z. Chen, and Y. Lu, “Gsemsplat: Generalizable Semantic 3D Gaussian Splatting From Uncalibrated Image Pairs,” preprint, arXiv:2412.16932 (2024).

[44] H. Zhu, T. Ding, T. Chen, I. Zharkov, R. Nevatia, and L. Liang, “Caesarnerf: Calibrated Semantic Representation for Few-Shot Generalizable Neural Rendering,” in European Conference on Computer Vision (Springer, 2024), 71-89.

[45] X. Sun, H. Jiang, L. Liu, et al. , Uni3r: Unified 3D Reconstruction and Semantic Understanding via Generalizable Gaussian Splatting From Unposed Multi-View Images (2025), https://arxiv.org/abs/2508.03643.

[46] H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual Instruction Tuning,” Advances in Neural Information Processing Systems 36 (2023): 34892-34916, https://doi.org/10.48550/arXiv.2304.08485.

[47] Z. Chen, J. Wu, W. Wang, et al., Internvl: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), 24185-24198.

[48] A. Radford, J. W. Kim, C. Hallacy, et al., Learning Transferable Visual Models From Natural Language Supervision,” in International Conference on Machine Learning (PmLR, 2021), 8748-8763.

[49] A. Masry, J. A. Rodriguez, T. Zhang, et al., Alignvlm: Bridging Vision and Language Latent Spaces for Multimodal Understanding,” preprint, arXiv:2502.01341 (2025).

[50] J. Xu, Z. Guo, J. He, et al., Qwen2.5-Omni Technical Report,” preprint, arXiv:2503.20215 (2025).

[51] S. Liu, Z. Zeng, T. Ren, et al., Grounding Dino: Marrying Dino With Grounded Pre-Training for Open-Set Object Detection,” in European Conference on Computer Vision (Springer, 2024), 38-55.

[52] S. Bai, K. Chen, X. Liu, et al., Qwen2.5-vl Technical Report,” preprint, arXiv:2502.13923 (2025).

[53] A. Kirillov, E. Mintun, N. Ravi, et al., Segment Anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), 4015-4026.

[54] J. L. Schönberger and J. -M. Frahm, “Structure-From-Motion Revisited,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 4104-4113, https://doi.org/10.1109/CVPR.2016.445.

[55] J. L. Schönberger, E. Zheng, J. M. Frahm, and M. Pollefeys, “Pixelwise View Selection for Unstructured Multi-View Stereo,” in Computer Vision - ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, ed. B. Leibe, J. Matas, N. Sebe, and M. Welling, Vol. 9907 (Springer, 2016), https://doi.org/10.1007/978-3-319-46487-9_31.

[56] M. Ye, M. Danelljan, F. Yu, and L. Ke, “Gaussian Grouping: Segment and Edit Anything in 3D Scenes,” in European Conference on Computer Vision (Springer, 2024), 162-179.