# Towards Online Continuous Sign Language Recognition and Translation
Code for Submission: Towards Online Continuous Sign Language Recognition and Translation


## Environment
Please run 
```
pip install -r requirements.txt
```


## Data Preparation
### CSLR Datasets
Phoenix-2014: Please follow https://www-i6.informatik.rwth-aachen.de/~koller/RWTH-PHOENIX/.
Phoenix-2014T: Please follow https://www-i6.informatik.rwth-aachen.de/~koller/RWTH-PHOENIX-2014-T/.
CSL-Daily: Please follow http://home.ustc.edu.cn/~zhouh156/dataset/csl-daily/.
Note that all raw videos need to be zipped.

### Keypoints
We use HRNet trained on COCO-WholeBody. Below is an example of extracting keypoints for the Phoenix-2014T training set.
```
cd ISLR
config_file='configs/phoenix-2014t_ISLR.yaml'
python -m torch.distributed.launch --nproc_per_node 8 --master_port 29999 --use_env gen_pose.py --config=${config_file} --split=train
```
The extracted keypoints are the same as those in TwoStream-SLR. Users can also check: https://github.com/FangyunWei/SLRT/tree/main/TwoStreamNetwork.

### S3D K400 pretrained model
Download S3D pretrained on K400 from https://github.com/kylemin/S3D.
Put it in ../pretrained_models/s3ds_actioncls_ckpt


## Online SLR Framework
Please check ISLR/README.md


## Online SLT
Please check Online_SLT/README.md


## Boosting an Offline Model with the Online Model
Please check CTC_based/README.md