You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+37-26Lines changed: 37 additions & 26 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,20 @@
1
-
# ASL Translation Data Preprocessing
1
+
# ASL Translation Data Preprocessing<!-- omit from toc -->
2
2
3
-
This repository follows the methodology described in the YouTube-ASL Dataset paper and provides a comprehensive solution for preprocessing American Sign Language (ASL) datasets, specifically designed to handle both **How2Sign** and **YouTube-ASL** datasets. Our preprocessing pipeline streamlines the workflow from video acquisition to landmark extraction, making the data ready for ASL translation tasks.
3
+
This repository provides a comprehensive solution for preprocessing American Sign Language (ASL) datasets, designed to handle both **How2Sign** and **YouTube-ASL** datasets. Our pipeline streamlines the workflow from video acquisition to landmark extraction, preparing the data for ASL translation tasks.
4
+
5
+
## Table of Contents<!-- omit from toc -->
6
+
7
+
-[Project Configuration](#project-configuration)
8
+
-[How to Use](#how-to-use)
9
+
-[YouTube-ASL](#youtube-asl)
10
+
-[How2Sign](#how2sign)
11
+
-[Dataset Introduction](#dataset-introduction)
12
+
-[YouTube-ASL Dataset](#youtube-asl-dataset)
13
+
-[How2Sign Dataset](#how2sign-dataset)
4
14
5
15
## Project Configuration
6
16
7
-
All project settings are centrally managed through `conf.py`, providing a single point of configuration for the entire preprocessing pipeline. Key configuration elements include:
17
+
All project settings are managed through `conf.py`, offering a single configuration point for the preprocessing pipeline. Key elements include:
8
18
9
19
-`ID`: Text file containing YouTube video IDs to process
10
20
-`VIDEO_DIR`: Directory for downloaded videos
@@ -17,38 +27,39 @@ All project settings are centrally managed through `conf.py`, providing a single
17
27
-`FRAME_SKIP`: Controls frame sampling rate for efficient processing
18
28
-`MAX_WORKERS`: Manages parallel processing to optimize performance
19
29
20
-
-`POSE_IDX`, `FACE_IDX`, `HAND_IDX`: Selected landmark indices for extracting the most relevant points for sign language analysis
21
-
22
-
This centralized approach allows easy adaptation to different hardware capabilities or dataset requirements without modifying the core processing code.
23
-
## How To Use?
24
-
-**YouTube-ASL**: make sure the constant is correct in conf.py. Then, operate step 1 to step 3.
25
-
-**How2Sign**: download **Green Screen RGB videos** and **English Translation (manually re-aligned)** from How2Sign website. Put the directory and .csv file in the right path or amend the path in the conf.py. then, operate step 3 only.
26
-
27
-
### Step 1: Data Acquisition (s1_data_downloader.py)
This step cleans text (converts Unicode characters, removes brackets), filters segments based on length and duration, and saves them with precise timestamps as tab-separated values.
-This step cleans text (converts Unicode characters, removes brackets), filters segments based on length and duration, and saves them with precise timestamps as tab-separated values.
- The script processes each video segment according to its timestamp, extracting only the most relevant body keypoints for sign language analysis. It uses parallel processing to handle multiple video efficiently. Results are saved as NumPy arrays.
The script processes each video segment according to its timestamp, extracting only the most relevant body keypoints for sign language analysis. Results are saved as NumPy arrays.
49
+
### How2Sign
50
+
1. Download **Green Screen RGB videos** and **English Translation (manually re-aligned)** from the How2Sign website.
51
+
2. Place the directory and .csv file in the correct path or amend the path in `conf.py`.
52
+
3. Run **Step 3: Feature Extraction** (`s3_mediapipe_labelling.py`) only.
42
53
43
54
## Dataset Introduction
44
55
45
56
### YouTube-ASL Dataset
46
-
Video List: [https://github.com/google-research/google-research/blob/master/youtube_asl/README.md](https://github.com/google-research/google-research/blob/master/youtube_asl/README.md)
47
-
Paper: ["YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpus" (Uthus et al., 2023)](https://arxiv.org/abs/2306.15162).
-**Paper**: ["YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpus" (Uthus et al., 2023)](https://arxiv.org/abs/2306.15162)
48
59
49
60
If you use YouTube-ASL, please cite their associated paper:
50
61
51
-
```
62
+
```bibtex
52
63
@misc{uthus2023youtubeasl,
53
64
author = {Uthus, David and Tanzer, Garrett and Georg, Manfred},
54
65
title = {YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpus},
@@ -60,12 +71,12 @@ If you use YouTube-ASL, please cite their associated paper:
Paper: [How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language](https://openaccess.thecvf.com/content/CVPR2021/html/Duarte_How2Sign_A_Large-Scale_Multimodal_Dataset_for_Continuous_American_Sign_Language_CVPR_2021_paper.html)
-**Paper**: [How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language](https://openaccess.thecvf.com/content/CVPR2021/html/Duarte_How2Sign_A_Large-Scale_Multimodal_Dataset_for_Continuous_American_Sign_Language_CVPR_2021_paper.html)
65
76
66
77
If you use How2Sign, please cite their associated paper:
67
78
68
-
```
79
+
```bibtex
69
80
@inproceedings{Duarte_CVPR2021,
70
81
title={{How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language}},
71
82
author={Duarte, Amanda and Palaskar, Shruti and Ventura, Lucas and Ghadiyaram, Deepti and DeHaan, Kenneth and
0 commit comments