VDNeRF: Vision-Only Dynamic Neural Radiance Field for Urban Scenes

Zhengyu Zou , Jingfeng Li , Hao Li , Xiaolei Hou , Jinwen Hu , Jingkun Chen , Lechao Cheng , Dingwen Zhang

CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (5) : 1585 -1599.

PDF (5354KB)
CAAI Transactions on Intelligence Technology ›› 2026, Vol. 11 ›› Issue (5) :1585 -1599. DOI: 10.1049/cit2.70086
ORIGINAL RESEARCH
research-article
VDNeRF: Vision-Only Dynamic Neural Radiance Field for Urban Scenes
Author information +
History +
PDF (5354KB)

Abstract

Neural radiance fields (NeRFs) implicitly model continuous three-dimensional scenes using a set of images with known camera poses, enabling the rendering of photorealistic novel views. However, existing NeRF-based methods encounter challenges in applications such as autonomous driving and robotic perception, primarily due to the difficulty of capturing accurate camera poses and limitations in handling large-scale dynamic environments. To address these issues, we propose vision-only dynamic NeRF (VDNeRF), a method that accurately recovers camera trajectories and learns spatiotemporal representations for dynamic urban scenes without requiring additional camera pose information or expensive sensor data. VDNeRF employs two separate NeRF models to jointly reconstruct the scene. The static NeRF model optimises camera poses and static background, whereas the dynamic NeRF model incorporates the 3D scene flow to ensure accurate and consistent reconstruction of dynamic objects. To address the ambiguity between camera motion and independent object motion, we design an effective and powerful training framework to achieve robust camera pose estimation and self-supervised decomposition of static and dynamic elements in a scene. Extensive evaluations on mainstream urban driving datasets demonstrate that VDNeRF surpasses state-of-the-art NeRF- based pose-free methods in both camera pose estimation and dynamic novel view synthesis.

Keywords

computer vision / deep neural networks / image processing / scene understanding

Cite this article

Download citation ▾
Zhengyu Zou, Jingfeng Li, Hao Li, Xiaolei Hou, Jinwen Hu, Jingkun Chen, Lechao Cheng, Dingwen Zhang. VDNeRF: Vision-Only Dynamic Neural Radiance Field for Urban Scenes. CAAI Transactions on Intelligence Technology, 2026, 11 (5) : 1585-1599 DOI:10.1049/cit2.70086

登录浏览全文

4963

注册一个新账户 忘记密码

References

[1]

B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “ NERF: Representing Scenes as Neural Radiance Fields for View Synthesis,” Communications of the ACM 65, no. 1 (2021): 99-106, https://doi.org/10.1145/3503250.

[2]

T.-M. Nguyen, S. Yuan, T. H. Nguyen, et al., “ Mcd: Diverse Large- Scale Multi-Campus Dataset for Robot Perception,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), 22304-22313.

[3]

J. Carmigniani and B. Furht, “ Augmented Reality: An Overview,” in Handbook of Augmented Reality (2011), 3-46.

[4]

G. C. Burdea and P. Coiffet, Virtual Reality Technology (John Wiley & Sons, 2003).

[5]

H. Li, D. Zhang, Y. Dai, et al., “ Gp-Nerf: Generalized Perception Nerf for Context-Aware 3d Scene Understanding,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2024), 21708-21718.

[6]

D. Zhang, H. Li, D. He, et al., “ Unsupervised Pre-Training With Language-Vision Prompts for Low-Data Instance Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence 47, no. 10 (2025): 8642-8657, https://doi.org/10.1109/tpami.2025.3579469.

[7]

J. Levinson, J. Askeland, J. Becker, et al., “ Towards Fully Autonomous Driving: Systems and Algorithms,” in 2011 IEEE Intelligent Vehicles Symposium (IV) (IEEE, 2011), 163-168.

[8]

Y. Jiao, Z. Jie, S. Chen, et al., “ Instance-Aware Multi-Camera 3d Object Detection With Structural Priors Mining and Self-Boosting Learning,” in Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 3 (2024): 2598-2606, https://doi.org/10.1609/aaai.v38i3.28037.

[9]

J. L. Schonberger and J.-M. Frahm, “ Structure-From-Motion Revisited,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2016), 4104-4113.

[10]

W. Bian, Z. Wang, K. Li, J.-W. Bian, and V. A. Prisacariu, “ Nope- Nerf: Optimising Neural Radiance Field With No Pose Prior,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), 4160-4169.

[11]

A. Meuleman, Y.-L. Liu, C. Gao, et al., “ Progressively Optimized Local Radiance Fields for Robust View Synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), 16539-16548.

[12]

Y.-L. Liu, C. Gao, A. Meuleman, et al., “ Robust Dynamic Radiance Fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), 13-23.

[13]

L. Yen-Chen, P. Florence, J. T. Barron, A. Rodriguez, P. Isola, and T.-Y. Lin, “ Inerf: Inverting Neural Radiance Fields for Pose Estimation,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2021), 1323-1330.

[14]

Z. Wang, S. Wu, W. Xie, M. Chen, and V. A. Prisacariu, “ Nerf-: Neural Radiance Fields Without Known Camera Parameters,” preprint arXiv:2102.07064 (2021).

[15]

Q. Meng, A. Chen, H. Luo, et al., “ Gnerf: Gan-Based Neural Radiance Field Without Posed Camera,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2021), 6351-6361.

[16]

C.-H. Lin, W.-C. Ma, A. Torralba, and S. Lucey, “ Barf: Bundle- Adjusting Neural Radiance Fields,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2021), 5741-5751.

[17]

Y. Jeong, S. Ahn, C. Choy, A. Anandkumar, M. Cho, and J. Park, “ Self-Calibrating Neural Radiance Fields,” in Proceedings of the IEEE/ CVF International Conference on Computer Vision (2021), 5846-5854.

[18]

Y. Chen, X. Chen, X. Wang, et al., “ Local-To-Global Registration for Bundle-Adjusting Neural Radiance Fields,” in Proceedings of the IEEE/ CVF Conference on Computer Vision and Pattern Recognition (2023), 8264-8273.

[19]

Q. Yan, Q. Wang, K. Zhao, et al., “ Cf-nerf: Camera Parameter Free Neural Radiance Fields With Incremental Learning,” in Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 6 (2024): 6440-6448, https://doi.org/10.1609/aaai.v38i6.28464.

[20]

Y. Ran, Y. Li, Q. Ye, et al., “ Ct-nerf: Incremental Optimizing Neural Radiance Field and Poses With Complex Trajectory,” preprint arXiv: 2404.13896 (2024).

[21]

D. Choi, H. Jang, and M. H. Kim, “ Omnilocalrf: Omnidirectional Local Radiance Fields From Dynamic Videos,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), 6871-6880.

[22]

Y. Gao, Y. Dai, H. Li, et al., “ Cosurfgs: Collaborative 3d Surface Gaussian Splatting With Distributed Learning for Large Scene Reconstruction,” preprint arXiv:2412.17612 (2024).

[23]

D. Wu, F. Yang, B. Xu, P. Liao, W. Zhao, and D. Zhang, “ Hv-bev: Decoupling Horizontal and Vertical Feature Sampling for Multi-View 3d Object Detection,” IEEE Transactions on Intelligent Transportation Systems 26, no. 11 (2025): 18734-18746, https://doi.org/10.1109/tits.2025.3590464.

[24]

R. Martin-Brualla, N. Radwan, M. S. Sajjadi, J. T. Barron, A. Dosovitskiy, and D. Duckworth, “ Nerf in the Wild: Neural Radiance Fields for Unconstrained Photo Collections,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021), 7210-7219.

[25]

A. Pumarola, E. Corona, G. Pons-Moll, and F. Moreno-Noguer, “ D- nerf: Neural Radiance Fields for Dynamic Scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021), 10318-10327.

[26]

K. Park, U. Sinha, J. T. Barron, et al., “ Nerfies: Deformable Neural Radiance Fields,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2021), 5865-5874.

[27]

K. Park, U. Sinha, P. Hedman, et al., “ Hypernerf: A Higher- Dimensional Representation for Topologically Varying Neural Radiance Fields,” preprint arXiv:2106.13228 40, no. 6 (2021): 1-12, https:// doi.org/10.1145/3478513.3480487.

[28]

Z. Li, S. Niklaus, N. Snavely, and O. Wang, “ Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021), 6498-6508.

[29]

C. Gao, A. Saraf, J. Kopf, and J.-B. Huang, “ Dynamic View Synthesis From Dynamic Monocular Video,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2021), 5712-5721.

[30]

T. Wu, F. Zhong, A. Tagliasacchi, F. Cole, and C. Oztireli, “ D 2nerf: Self-Supervised Decoupling of Dynamic and Static Objects From a Monocular Video,” Advances in Neural Information Processing Systems 35 (2022): 32653-32666, https://doi.org/10.5555/3600270.3602636.

[31]

H. Sun, X. Li, L. Shen, X. Ye, K. Xian, and Z. Cao, “ Dyblurf: Dynamic Neural Radiance Fields From Blurry Monocular Video,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), 7517-7527.

[32]

Y. Xiangli, L. Xu, X. Pan, et al., “ Bungeenerf: Progressive Neural Radiance Field for Extreme Multi-Scale Scene Rendering,” in European Conference on Computer Vision (Springer, 2022), 106-122.

[33]

M. Tancik, V. Casser, X. Yan, et al., “ Block-Nerf: Scalable Large Scene Neural View Synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), 8248-8258.

[34]

H. Turki, D. Ramanan, and M. Satyanarayanan, “ Mega-Nerf: Scalable Construction of Large-Scale Nerfs for Virtual Fly-Throughs,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), 12922-12931.

[35]

Z. Xie, J. Zhang, W. Li, F. Zhang, and L. Zhang, “ S-Nerf: Neural Radiance Fields for Street Views,” preprint arXiv:2303.00749 (2023).

[36]

J. Guo, N. Deng, X. Li, et al., “ Streetsurf: Extending Multi-View Implicit Surface Reconstruction to Street Views,” preprint arXiv: 2306.04988 (2023).

[37]

T. Yuan, Y. Mao, J. Yang, Y. Liu, Y. Wang, and H. Zhao, “ Presight: Enhancing Autonomous Vehicle Perception With City-Scale Nerf Priors,” preprint arXiv:2403.09079 (2024): 323-339, https://doi.org/10.1007/978-3-031-72980-5_19.

[38]

H. Turki, J. Y. Zhang, F. Ferroni, and D. Ramanan, “ Suds: Scalable Urban Dynamic Scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), 12375-12385.

[39]

J. Yang, B. Ivanovic, O. Litany, et al., “ Emernerf: Emergent Spatial- Temporal Scene Decomposition via Self-supervision,” preprint arXiv: 2311.02077 (2023).

[40]

T.-A.-Q. Nguyen, L. Roldão, N. Piasco, M. Bennehar, and D. Tsishkou, “ Rodus: Robust Decomposition of Static and Dynamic Elements in Urban Scenes,” preprint arXiv:2403.09419 (2024): 112-130, https://doi.org/10.1007/978-3-031-73220-1_7.

[41]

J. T. Kajiya and B. P. Von Herzen, “ Ray Tracing Volume Densities,” ACM SIGGRAPH computer graphics 18, no. 3 (1984): 165-174, https:// doi.org/10.1145/800031.808594.

[42]

T. Müller, A. Evans, C. Schied, and A. Keller, “ Instant Neural Graphics Primitives With a Multiresolution Hash Encoding,” ACM Transactions on Graphics 41, no. 4 (2022): 1-15, https://doi.org/10.1145/3528223.3530127.

[43]

K. Zhang, G. Riegler, N. Snavely, and V. Koltun, “ Nerf++: Analyzing and Improving Neural Radiance Fields,” preprint arXiv: 2010.07492, (2020).

[44]

R. Ranftl, A. Bochkovskiy, and V. Koltun, “ Vision Transformers for Dense Prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (2021), 12179-12188.

[45]

K. Deng, A. Liu, J.-Y. Zhu, and D. Ramanan, “ Depth-Supervised Nerf: Fewer Views and Faster Training for Free,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), 12882-12891.

[46]

Z. Teed and J. Deng, “ Raft: Recurrent All-Pairs Field Transforms for Optical Flow,” in Computer Vision-ECCV 2020: 16Th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16 (Springer, 2020), 402-419.

[47]

R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun, “ Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer,” IEEE Transactions on Pattern Analysis and Machine Intelligence 44, no. 3 (2020): 1623-1637, https:// doi.org/10.1109/tpami.2020.3019967.

[48]

P. Xiao, Z. Shao, S. Hao, et al., “ Pandaset: Advanced Sensor Suite Dataset for Autonomous Driving,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) (IEEE, 2021), 3095-3101.

[49]

P. Sun, H. Kretzschmar, X. Dotiwalla, et al., “ Scalability in Perception for Autonomous Driving: Waymo Open Dataset,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020), 2446-2454.

[50]

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “ Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing 13, no. 4 (2004): 600-612, https://doi.org/10.1109/tip.2003.819861.

[51]

R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “ The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018), 586-595.

[52]

S. Umeyama, “ Least-Squares Estimation of Transformation Parameters Between Two Point Patterns,” IEEE Transactions on Pattern Analysis and Machine Intelligence 13, no. 4 (1991): 376-380, https://doi.org/10.1109/34.88573.

[53]

T. Müller, Tiny Cuda Neural Network Framework. Retrieved May 18, (2021) 2023.

[54]

R. Li, M. Tancik, and A. Kanazawa, “ Nerfacc: A General Nerf Acceleration Toolbox,” preprint arXiv:2210.04847 (2022).

[55]

D. P. Kingma and J. Ba, “ Adam: A Method for Stochastic Optimization,” preprint arXiv:1412.6980 (2014).

[56]

J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman, “ Mip-Nerf 360: Unbounded Anti-Aliased Neural Radiance Fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), 5470-5479.

[57]

D. Cheng, Y. Ji, D. Gong, et al., “ Continual All-in-One Adverse Weather Removal With Knowledge Replay on a Unified Network Structure,” IEEE Transactions on Multimedia 26 (2024): 8184-8196, https://doi.org/10.1109/tmm.2024.3377136.

[58]

D. Cheng, Y. Hu, N. Wang, D. Zhang, and X. Gao, “ Achieving Plasticity-Stability Trade-Off in Continual Learning Through Adaptive Orthogonal Projection,” IEEE Transactions on Circuits and Systems for Video Technology 35, no. 8 (2025): 7485-7498, https://doi.org/10.1109/tcsvt.2025.3547916.

PDF (5354KB)

0

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉