Sign language data quality improvement based on dual information streams

Abstract

Sign language dataset is essential in sign language recognition and translation (SLRT). Current public sign language datasets are small and lack diversity, which does not meet the practical application requirements for SLRT. However, making a large-scale and diverse sign language dataset is difficult as sign language data on the Internet is scarce. In making a large-scale and diverse sign language dataset, some sign language data qualities are not up to standard. This paper proposes a two information streams transformer (TIST) model to judge whether the quality of sign language data is qualified. To verify that TIST effectively improves sign language recognition (SLR), we make two datasets, the screened dataset and the unscreened dataset. In this experiment, this paper uses visual alignment constraint (VAC) as the baseline model. The experimental results show that the screened dataset can achieve better word error rate (WER) than the unscreened dataset.

RIGHTS & PERMISSIONS

Tianjin University of Technology