We release a continuous fingerspelling resource for Indian Sign Language, comprising 1,308 video segments with aligned text. Building on this, we add character-level frame annotations for every letter and an automatically discovered set of 11,700+ additional segments, supporting research in fingerspelling recognition, detection, temporal localization, and sign generation.
The first continuous fingerspelling dataset for ISL: 1,308 segments from 499 ISH News videos with aligned word-level text. A ByT5-small baseline reaches 82.91% CER after fine-tuning.
Extends ISL-FS with character-level frame boundaries for every letter in all 1,308 segments (99.1% expert-validated), and 11,700+ auto-discovered segments mined from the iSign corpus. Dense supervision reaches 4.87% CER, beating word-level baselines (16.4%) and BSL transfer (8.1%); the same model also yields 84.6% F1 fingerspelling detection without dedicated detection training.
Boundaries follow a transition-onset rule: marked at the frame where the hand begins moving toward the next letter shape. A professional ISL interpreter blindly re-transcribed 660 cropped letter segments; only 6 were genuine errors, giving 99.1% validated accuracy.
| Method | Input | Supervision | Standard CER | Signer-Indep CER |
|---|---|---|---|---|
| ByT5 | RGB | Word | 85.9% | 81.4% |
| Transpeller (BSL pretrained) | RGB | Word | 8.1% | 15.7% |
| ResNet-BiLSTM | RGB | Word | 16.4% | 37.4% |
| ResNet-BiLSTM | RGB | Frame + Word | 4.87% | 16.8% |
| MLP-BiLSTM | Keypoint | Frame + Word | 5.1% | 14.3% |
Two pipelines mine the iSign corpus: a multi-scale sliding window (7,833 segments) and detection-based matching against NER-extracted proper nouns (7,479 segments, 51.7% non-overlapping). Adding them to training improves signer-independent CER by 10% relative on RGB and 6% on keypoints.
ISL-Fingerspelling/ ├── videos/ # 1,308 MP4 video files ├── fingerspelling_annotations.csv # Segment annotations ├── localization_timestamps.csv # Temporal localization in source videos └── README.md
fingerspelling_annotations.csv: uid (format {video_id}_seg{index}), text (fingerspelled text)localization_timestamps.csv: video_id, segment_index, start_time/start_sec, end_time/end_sec, duration_str/duration_sec, transcriptThe ISL-FS-Dense release will include character-level frame boundaries for every letter in the 1,308 manually annotated segments, plus the 11,700+ automatically discovered segments with their matched transcripts. Dataset link will be added here once published.
If you use either release in your research, please cite the relevant paper(s):
@inproceedings{r-etal-2025-continuous,
title = {Continuous Fingerspelling Dataset for {I}ndian {S}ign {L}anguage},
author = {R, Kirandevraj and Kurmi, Vinod K. and Namboodiri, Vinay P. and Jawahar, C.V.},
booktitle = {Proceedings of the Workshop on Sign Language Processing (WSLP)},
publisher = {Association for Computational Linguistics},
year = {2025},
pages = {33--38},
url = {https://aclanthology.org/2025.wslp-main.6/},
doi = {10.18653/v1/2025.wslp-main.6}
}
@inproceedings{kirandevraj2026denseisl,
title={Dense Frame Annotations for Low-Resource ISL Fingerspelling Recognition},
author={Kirandevraj, R and Kurmi, Vinod K and Namboodiri, Vinay P and Jawahar, CV},
booktitle={Proceedings of the International Conference on Pattern Recognition (ICPR)},
year={2026}
}
Data sourced from publicly available ISH News YouTube videos. 407 of 499 videos overlap with the iSign dataset. Both datasets are released under CC BY-NC 4.0 license for research purposes only.