SIGN LANGUAGUE RECOGNITION MODELS

          SubUNets:

• The system is  series of specialized systems, termed SubUNets with each SubUNet processes the frames of the video independently.
• Allow to inject domain-specific expert knowledge into the system regarding suitable intermediate representations.
• Allow to implicitly perform transfer learning between different interrelated tasks.

SubUNets: End-to-end Hand Shape and Continuous Sign Language Recognition

Reference: N. C. Camgoz, S. Hadfield, O. Koller, and R. Bowden, “Subunets: Endto-end hand shape and continuous sign language recognition,” in 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017, pp. 3075–3084

• One of the primary information carrying modalities in sign language is the hand shape.

Combination of Hand SubUnets and Full Framed SubUnets:

Iterative Alignment Network for Continuous Sign Language Recognition

3D-ResNet+BLSTM

• Feature extractor : 3D Resnet
• BLSTM encoder for sequence learning.
• Decoding Strategies : CTC criterion and attention decoder RNN.
• Soft DTW: similarity between various length sequences , also helps in warping path i.e. possible alignment between sign clip and words.

Reference:  https://ieeexplore.ieee.org/document/8954236