深度学习>> SLR>> 返回
项目作者： 0aqz0

项目描述 ：
isolated & continuous sign language recognition using CNN+LSTM/3D CNN/GCN/Encoder-Decoder
高级语言： Python
项目地址: git://github.com/0aqz0/SLR.git
创建时间： 2019-12-03T08:49:02Z
项目社区：https://github.com/0aqz0/SLR
开源协议：
下载


SLR

isolated & continuous sign language recognition using CNN+LSTM/3D CNN/GCN/Encoder-Decoder

Requirements

Isolated Sign Language Recognition

CNN+LSTM

  1. four layers of Conv2d + one layer of LSTM

    | Dataset | Classes | Samples | Best Test Acc | Best Test Loss |
    | —————— | ———- | ———- | ——————- | ——————— |
    | CSL_Isolated | 100 | 25,000 | 82.08% | 0.734426 |
    | CSL_Isolated | 500 | 125,000 | 71.71% | 1.332122 |

  2. ResNet + one layer of LSTM

    | Dataset | Classes | Samples | Best Test Acc | Best Test Loss |
    | —————— | ———- | ———- | ——————- | ——————— |
    | CSL_Isolated | 100 | 25,000 | 93.54% | 0.245582 |
    | CSL_Isolated | 500 | 125,000 | 83.17% | 0.748759 |

3D CNN

  1. three layers of Conv3d

    | Dataset | Classes | Samples | Best Test Acc | Best Test Loss |
    | —————— | ———- | ———- | ——————- | ——————— |
    | CSL_Isolated | 100 | 25,000 | 58.86% | 1.560049 |
    | CSL_Isolated | 500 | 125,000 | 45.07% | 2.255563 |

  2. 3D ResNet

    | Method | Dataset | Classes | Samples | Best Test Acc | Best Test Loss |
    | ————- | —————— | ———- | ———- | ——————- | ——————— |
    | ResNet18 | CSL_Isolated | 100 | 25,000 | 93.30% | 0.246169 |
    | ResNet18 | CSL_Isolated | 500 | 125,000 | 79.42% | 0.800490 |
    | ResNet34 | CSL_Isolated | 100 | 25,000 | 94.78% | 0.207592 |
    | ResNet34 | CSL_Isolated | 500 | 125,000 | 81.61% | 0.750424 |
    | ResNet50 | CSL_Isolated | 100 | 25,000 | 94.36% | 0.232631 |
    | ResNet50 | CSL_Isolated | 500 | 125,000 | 83.15% | 0.803212 |
    | ResNet101 | CSL_Isolated | 100 | 25,000 | 95.26% | 0.205430 |
    | ResNet101 | CSL_Isolated | 500 | 125,000 | 83.18% | 0.751727 |

  3. ResNet (2+1)D

    | Dataset | Classes | Samples | Best Test Acc | Best Test Loss |
    | —————— | ———- | ———- | ——————- | ——————— |
    | CSL_Isolated | 100 | 25,000 | 98.68% | 0.043099 |
    | CSL_Isolated | 500 | 125,000 | 94.85% | 0.234880 |

GCN

Dataset Classes Samples Best Test Acc Best Test Loss
CSL_Skeleton 100 25,000 79.20% 0.737053
CSL_Skeleton 500 125,000 66.64% 1.165872

Skeleton+LSTM

Dataset Classes Samples Best Test Acc Best Test Loss
CSL_Skeleton 100 25,000 84.30% 0.488253
CSL_Skeleton 500 125,000 70.62% 1.078730

Continuous Sign Language Recognition

Encoder-Decoder

Encoder is ResNet18+LSTM, and Decoder is LSTM

Dataset Sentences Samples Best Test Wer Best Test Loss
CSL_Continuous 100 25,000 1.01% 0.034636
CSL_Continuous_Char 100 25,000 1.19% 0.049449

References