Human motion recognition using ultra-wideband radar and cameras on mobile robot

Abstract

Cameras can reliably detect human motions in a normal environment, but they are usually affected by sudden illumination changes and complex conditions, which are the major obstacles to the reliability and robustness of the system. To solve this problem, a novel integration method was proposed to combine bi-static ultra-wideband radar and cameras. In this recognition system, two cameras are used to localize the object’s region, regions while a radar is used to obtain its 3D motion models on a mobile robot. The recognition results can be matched in the 3D motion library in order to recognize its motions. To confirm the effectiveness of the proposed method, the experimental results of recognition using vision sensors and those of recognition using the integration method were compared in different environments. Higher correct-recognition rate is achieved in the experiment.

Keywords

ultra-wideband radar / computer vision / pattern recognition / human motion / mobile robot

References

[1] Yamato J., Ohya J., Ishii K. Recognizing human action in time-sequential images using hidden Markov model[C] Proc 1992 IEEE Computer Society Conf Computer Vision and Pattern Recognition, 1992, Champaign: IEEE 379-385.

[2] Wilson A. D., Bobick A. F., Cassell J. Recovering the temporal structure of natural gesture[C] Proc 2nd Int Conf Automatic Face and Gesture Recognition, 1996, California: IEEE 66-71.

[3] Yang J., Xu Y., Chen C. S. Human action learning via hidden Markov model[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1997, 27(1): 34-44.

[4] Brand M., Oliver N., Pentland A. Coupled Hidden Markov Models for Complex Action Recognition[R]. 1996, USA: Media Lab Vision and Modelling, MIT.

[5] Brand M., Oliver N., Pentland A. Coupled hidden Markov models for complex action recognition[C] Proc 1997 IEEE Computer Society Conf Computer Vision and Pattern Recognition, 1997, San Juan: IEEE 994-999.

[6] Wilson A. D., Bobick A. F. Parametric hidden Markov models for gesture recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999, 21(9): 884-900.

[7] Galata A., Johnson N., Hogg D. Learning behaviour models of human activities[C] British Machine Vision Conf Modelling Human Behaviour, 1999, Nottingham: British Machine Vision Association 12-22.

[8] Galata A., Johnson N., Hogg D. Learning structured behavior models using variable length Markov models[C] IEEE Int Workshop on Modelling People, 1999, Greece: IEEE 95-102.

[9] Brand M., Kettnaker V. Discovery and segmentation of activities in video[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(8): 844-851.

[10] Horvitz E., Oliver N., Garg A. Layered representations for human activity recognition[C] Proceedings of the 4th IEEE Int Conf Multimodal Interfaces, 2002, Greece: IEEE 3-8.

[11] Mori T., Segawa Y., Shimosaka M., et al. Hierarchical recognition of daily human actions based on continuous hidden Markov models[C] Automatic Face and Gesture Recognition, 2004, Seoul: IEEE 779-784.

[12] Garg A., Oliver N., Horvitz E. Layered representations for learning and inferring office activity from multiple sensory channels[J]. Computer Vision and Image Understanding, Event Detection in Video, 2004, 96(2): 163-180.

[13] Ramanan D., Forsyth D. A., Zisserman A. Tracking people by learning their appearance[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(1): 65-81.

[14] Han J., Nguyen C. Development of a tunable multiband UWB radar sensor and its applications to subsurface sensing[J]. IEEE Sensors Journal, 2007, 7(1): 51-58.

[15] Mita T., Kaneko T., Hori O. Joint haar-like features for face detection[C] Tenth IEEE Int Conf Computer Vision, 2005, Beijing: IEEE 1619-1626.

[16] Demirkir C., Sankur B. Face detection using boosted tree classifier stages[C] Proc of the IEEE 12th Signal Processing and Communications Applications Conf, 2004, Istanbul, Turkey: IEEE 575-578.

[17] Kidera S., Kani Y., Sakamoto T., et al. An experimental study for a high-resolution 3-D imaging algorithm with linear array for UWB radars[C] IEEE Int Conf Ultra-Wideband, 2007, Singapore: IEEE 600-605.

[18] Sakamoto T., Sato T. Real-time imaging of human bodies with UWB radars using walking motion[C] IEEE Int Conf Ultra-Wideband, 2007, Singapore: IEEE 26-30.

[19] Ohtake Y., Belyaev A., Seidel H. P. 3D scattered data interpolation and approximation with multilevel compactly supported RBFs[J]. Graph Models, 2005, 67(2): 150-165.

[20] Arikan O., Forsyth D. A., O’Brien J. Motion synthesis from annotations[J]. ACM Transactions on Graphics, 2003, 22(3): 402-408.

[21] Arikan O., Forsyth D. A. Interactive motion generation from examples[J]. ACM Transactions on Graphics, 2002, 21(3): 483-490.