Multiple Sign Language Identification Using Deep Learning Techniques

Ahmed Mahmoud Sultan; Waleed Makram Mohamed Zaki; Mohammed Kayed; Abdel Mgeid Amin Ali

doi:doi:10.11648/j.cssp.20231101.11

| Peer-Reviewed

Multiple Sign Language Identification Using Deep Learning Techniques

Ahmed Mahmoud Sultan, Waleed Makram Mohamed Zaki, Mohammed Kayed, Abdel Mgeid Amin Ali

Published in Science Journal of Circuits, Systems and Signal Processing (Volume 11, Issue 1)

Received: 21 January 2023 Accepted: 14 February 2023 Published: 29 May 2023

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

The research presents a general overview of sign languages, and a previous survey was conducted on all aspects of sign languages including the tools used to collect sign languages and the best algorithms to achieve the best results. A specialized database is prepared to combine the alphabet signs of the Arabic, American, and British languages, as they are the most important sign languages and the most widespread in the world. Based on different sign languages and deep learning techniques such as LeNet, VGG-16, and CapsNet, which are considered among the best methods for solving sign language problems based on our previous studies. The purpose of the research is to remove the communication gap between the deaf, and normal speaking people who speak one sign language or those who try to communicate from different countries and to identify these languages easily. We applied some of the traditional deep learning techniques such as LeNet, and then we applied VGG-16 using pre-training models and adjusted some layers to suit our problem. Also we applied CapsNet as it is perfectly suitable for solving the problem of sign language deformation, rotation, and scaling. The best results were achieved using VGG-16, as it was trained on a previous database like ImageNet, which contains millions of images. We got an accuracy of 99.69% when training the model of VGG-16, and an accuracy of 99.65% when testing the model. On the other hand, we got lower accuracies in CapsNet and LeNet compared to VGG-16. We got 96.54%, 97.45%, and 94.95% on BSL, ASL, and ArSL respectively while applying LeNet model, while we got 98.4848%, 98.4286%, and 99.5652% on ArSL, ASL, and BSL respectively while applying CapsNet model. Using VGG-16 we got 99.05%, 98.50%, and 99.69% on ArSL, ASL, and BSL respectively.

Published in	Science Journal of Circuits, Systems and Signal Processing (Volume 11, Issue 1)
DOI	10.11648/j.cssp.20231101.11
Page(s)	1-11
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2023. Published by Science Publishing Group

Keywords

SLI, Capsule Neural Network, Deep Learning, VGG16, LeNet

References

[1]	J. Quer and M. Steinbach, "Handling sign language data: The impact of modality," Frontiers in Psychology, 2019.
[2]	R. Kushalnagar, "Deafness and hearing loss," Human–Computer Interaction Series, pp. 35-47, 2019.
[3]	B. I. II, "Deafness in the Arab world: a general investigation, with applications to Lebanon," scholarship.tricolib.brynmawr.edu, 2018.
[4]	C. D. Monteiro, C. M. Mathew, R. Gutierrez-Osuna and F. Shipman, "Detecting and identifying sign languages through visual features," 2016 IEEE International Symposium on Multimedia (ISM), 2016.
[5]	A. Sultan, W. Makram, M. Kayed and A. A. ALi, "Sign language identification and recognition: A comparative study," Open Computer Science, pp. 191-210, 2022.
[6]	M. Mohandes, Junzhao Liu and M. Deriche, "A survey of image-based Arabic sign language recognition," 2014 IEEE 11th International Multi-Conference on Systems, Signals & Devices (SSD14), 2014.
[7]	R. Daroya, D. Peralta and P. Naval, "Alphabet sign language image classification using deep learning," TENCON 2018 - 2018 IEEE Region 10 Conference, 2018.
[8]	E. Goceri, "Analysis of capsule networks for Image Classification," Proceedings of the 15th International Conference on Computer Graphics, Visualization, Computer Vision and Image Processing (CGVCVIP 2021), the 7th International Conference on Connected Smart Cities (CSC 2021) and 6th International Conference on Big Data A, 2021.
[9]	S. Sabour, N. Frosst and H. Geoffrey E, "Dynamic routing between capsules," Advances in neural information processing systems, 2017.
[10]	Y. LeCun, B. Yoshua and H. Geoffrey, "Deep learning," nature, vol. 521, no. 7553, pp. 436-444, 2015.
[11]	A. El Zaar, N. Benaya and A. El Allati, "Sign language recognition: High performance deep learning approach applyied to multiple sign languages," E3S Web of Conferences, 2022.
[12]	H. T. Nguyen, L. T. Pham, T. T. Mai, T. K. Vo and T. T. Dien, "Letter recognition in hand sign language with VGG-16," Intelligent Systems and Networks, pp. 410-417, 2022.
[13]	Z. Alsaadi, E. Alshamani, M. Alrehaili, A. A. Alrashdi, S. Albelwi and A. O. Elfaki, "A real time Arabic Sign Language Alphabets (arsla) recognition model using deep learning architecture," Computers, vol. 11, no. 5, p. 78, 2022.
[14]	A.-J. Tanseem N and A.-J. Abu-Jamie, "Classification of Sign-language Using VGG16," 2022.
[15]	V. Shreya and Y. Shaik Sohail, "Sign Language Interpreter Using Computer Vision and LeNet-5 Convolutional Neural Network Architecture," International Journal of Innovative Science and Research Technolog, vol. 6, 2021.
[16]	M. Bilgin and K. Mutludogan, "American sign language character recognition with capsule networks," 2019 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), 2019.
[17]	H. Xiao, Y. Yang, K. Yu, J. Tian, X. Cai, U. Muhammad and J. Chen, "Sign language digits and alphabets recognition by Capsule Networks," Journal of Ambient Intelligence and Humanized Computing, vol. 13, no. 4, pp. 2131-2141, 2021.
[18]	A. M. B. S. A. K. M. B. and M., "Different techniques of hand segmentation in the real time," IJCAIT, pp. 45-49, 2013.
[19]	Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard and L. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Computation, pp. 541-551, 1989.
[20]	S. Karen and Z. Andrew, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv: 1409.1556, 2014.
[21]	G. B and S. Natarajan, "Hyperparameter optimisation for Capsule Networks," EAI Endorsed Transactions on Cloud Systems, 2019.
[22]	J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li and Li Fei-Fei, "ImageNet: A large-scale hierarchical image database," 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009.
[23]	K. Suri and R. Gupta, "Continuous sign language recognition from wearable Imus using deep capsule networks and game theory," Computers & Electrical Engineering, vol. 78, pp. 493-503, 2019.

Cite This Article

Plain Text BibTeX RIS

APA Style

Ahmed Mahmoud Sultan, Waleed Makram Mohamed Zaki, Mohammed Kayed, Abdel Mgeid Amin Ali. (2023). Multiple Sign Language Identification Using Deep Learning Techniques. Science Journal of Circuits, Systems and Signal Processing, 11(1), 1-11. https://doi.org/10.11648/j.cssp.20231101.11

Copy | Download

ACS Style

Ahmed Mahmoud Sultan; Waleed Makram Mohamed Zaki; Mohammed Kayed; Abdel Mgeid Amin Ali. Multiple Sign Language Identification Using Deep Learning Techniques. Sci. J. Circuits Syst. Signal Process. 2023, 11(1), 1-11. doi: 10.11648/j.cssp.20231101.11

Copy | Download

AMA Style

Ahmed Mahmoud Sultan, Waleed Makram Mohamed Zaki, Mohammed Kayed, Abdel Mgeid Amin Ali. Multiple Sign Language Identification Using Deep Learning Techniques. Sci J Circuits Syst Signal Process. 2023;11(1):1-11. doi: 10.11648/j.cssp.20231101.11

Copy | Download

@article{10.11648/j.cssp.20231101.11,
  author = {Ahmed Mahmoud Sultan and Waleed Makram Mohamed Zaki and Mohammed Kayed and Abdel Mgeid Amin Ali},
  title = {Multiple Sign Language Identification Using Deep Learning Techniques},
  journal = {Science Journal of Circuits, Systems and Signal Processing},
  volume = {11},
  number = {1},
  pages = {1-11},
  doi = {10.11648/j.cssp.20231101.11},
  url = {https://doi.org/10.11648/j.cssp.20231101.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.cssp.20231101.11},
  abstract = {The research presents a general overview of sign languages, and a previous survey was conducted on all aspects of sign languages including the tools used to collect sign languages and the best algorithms to achieve the best results. A specialized database is prepared to combine the alphabet signs of the Arabic, American, and British languages, as they are the most important sign languages and the most widespread in the world. Based on different sign languages and deep learning techniques such as LeNet, VGG-16, and CapsNet, which are considered among the best methods for solving sign language problems based on our previous studies. The purpose of the research is to remove the communication gap between the deaf, and normal speaking people who speak one sign language or those who try to communicate from different countries and to identify these languages easily. We applied some of the traditional deep learning techniques such as LeNet, and then we applied VGG-16 using pre-training models and adjusted some layers to suit our problem. Also we applied CapsNet as it is perfectly suitable for solving the problem of sign language deformation, rotation, and scaling. The best results were achieved using VGG-16, as it was trained on a previous database like ImageNet, which contains millions of images. We got an accuracy of 99.69% when training the model of VGG-16, and an accuracy of 99.65% when testing the model. On the other hand, we got lower accuracies in CapsNet and LeNet compared to VGG-16. We got 96.54%, 97.45%, and 94.95% on BSL, ASL, and ArSL respectively while applying LeNet model, while we got 98.4848%, 98.4286%, and 99.5652% on ArSL, ASL, and BSL respectively while applying CapsNet model. Using VGG-16 we got 99.05%, 98.50%, and 99.69% on ArSL, ASL, and BSL respectively.},
 year = {2023}
}

Copy | Download

TY  - JOUR
T1  - Multiple Sign Language Identification Using Deep Learning Techniques
AU  - Ahmed Mahmoud Sultan
AU  - Waleed Makram Mohamed Zaki
AU  - Mohammed Kayed
AU  - Abdel Mgeid Amin Ali
Y1  - 2023/05/29
PY  - 2023
N1  - https://doi.org/10.11648/j.cssp.20231101.11
DO  - 10.11648/j.cssp.20231101.11
T2  - Science Journal of Circuits, Systems and Signal Processing
JF  - Science Journal of Circuits, Systems and Signal Processing
JO  - Science Journal of Circuits, Systems and Signal Processing
SP  - 1
EP  - 11
PB  - Science Publishing Group
SN  - 2326-9073
UR  - https://doi.org/10.11648/j.cssp.20231101.11
AB  - The research presents a general overview of sign languages, and a previous survey was conducted on all aspects of sign languages including the tools used to collect sign languages and the best algorithms to achieve the best results. A specialized database is prepared to combine the alphabet signs of the Arabic, American, and British languages, as they are the most important sign languages and the most widespread in the world. Based on different sign languages and deep learning techniques such as LeNet, VGG-16, and CapsNet, which are considered among the best methods for solving sign language problems based on our previous studies. The purpose of the research is to remove the communication gap between the deaf, and normal speaking people who speak one sign language or those who try to communicate from different countries and to identify these languages easily. We applied some of the traditional deep learning techniques such as LeNet, and then we applied VGG-16 using pre-training models and adjusted some layers to suit our problem. Also we applied CapsNet as it is perfectly suitable for solving the problem of sign language deformation, rotation, and scaling. The best results were achieved using VGG-16, as it was trained on a previous database like ImageNet, which contains millions of images. We got an accuracy of 99.69% when training the model of VGG-16, and an accuracy of 99.65% when testing the model. On the other hand, we got lower accuracies in CapsNet and LeNet compared to VGG-16. We got 96.54%, 97.45%, and 94.95% on BSL, ASL, and ArSL respectively while applying LeNet model, while we got 98.4848%, 98.4286%, and 99.5652% on ArSL, ASL, and BSL respectively while applying CapsNet model. Using VGG-16 we got 99.05%, 98.50%, and 99.69% on ArSL, ASL, and BSL respectively.
VL  - 11
IS  - 1
ER  -

Copy | Download

Author Information

Ahmed Mahmoud Sultan

Computer Science Department, Faculty of Computers and Artificial Intelligence, Beni-Suef, Egypt
Waleed Makram Mohamed Zaki

Information System Department, Faculty of Computers and Information Systems, Minia, Egypt
Mohammed Kayed

Computer Science Department, Faculty of Computers and Artificial Intelligence, Beni-Suef, Egypt
Abdel Mgeid Amin Ali

Computer Science Department, Faculty of Computers and Information Systems, Minia, Egypt

Download PDF

Sections

Plain Text BibTeX RIS

APA Style

Ahmed Mahmoud Sultan, Waleed Makram Mohamed Zaki, Mohammed Kayed, Abdel Mgeid Amin Ali. (2023). Multiple Sign Language Identification Using Deep Learning Techniques. Science Journal of Circuits, Systems and Signal Processing, 11(1), 1-11. https://doi.org/10.11648/j.cssp.20231101.11

Copy | Download

ACS Style

Ahmed Mahmoud Sultan; Waleed Makram Mohamed Zaki; Mohammed Kayed; Abdel Mgeid Amin Ali. Multiple Sign Language Identification Using Deep Learning Techniques. Sci. J. Circuits Syst. Signal Process. 2023, 11(1), 1-11. doi: 10.11648/j.cssp.20231101.11

Copy | Download

AMA Style

Ahmed Mahmoud Sultan, Waleed Makram Mohamed Zaki, Mohammed Kayed, Abdel Mgeid Amin Ali. Multiple Sign Language Identification Using Deep Learning Techniques. Sci J Circuits Syst Signal Process. 2023;11(1):1-11. doi: 10.11648/j.cssp.20231101.11

Copy | Download

@article{10.11648/j.cssp.20231101.11,
  author = {Ahmed Mahmoud Sultan and Waleed Makram Mohamed Zaki and Mohammed Kayed and Abdel Mgeid Amin Ali},
  title = {Multiple Sign Language Identification Using Deep Learning Techniques},
  journal = {Science Journal of Circuits, Systems and Signal Processing},
  volume = {11},
  number = {1},
  pages = {1-11},
  doi = {10.11648/j.cssp.20231101.11},
  url = {https://doi.org/10.11648/j.cssp.20231101.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.cssp.20231101.11},
  abstract = {The research presents a general overview of sign languages, and a previous survey was conducted on all aspects of sign languages including the tools used to collect sign languages and the best algorithms to achieve the best results. A specialized database is prepared to combine the alphabet signs of the Arabic, American, and British languages, as they are the most important sign languages and the most widespread in the world. Based on different sign languages and deep learning techniques such as LeNet, VGG-16, and CapsNet, which are considered among the best methods for solving sign language problems based on our previous studies. The purpose of the research is to remove the communication gap between the deaf, and normal speaking people who speak one sign language or those who try to communicate from different countries and to identify these languages easily. We applied some of the traditional deep learning techniques such as LeNet, and then we applied VGG-16 using pre-training models and adjusted some layers to suit our problem. Also we applied CapsNet as it is perfectly suitable for solving the problem of sign language deformation, rotation, and scaling. The best results were achieved using VGG-16, as it was trained on a previous database like ImageNet, which contains millions of images. We got an accuracy of 99.69% when training the model of VGG-16, and an accuracy of 99.65% when testing the model. On the other hand, we got lower accuracies in CapsNet and LeNet compared to VGG-16. We got 96.54%, 97.45%, and 94.95% on BSL, ASL, and ArSL respectively while applying LeNet model, while we got 98.4848%, 98.4286%, and 99.5652% on ArSL, ASL, and BSL respectively while applying CapsNet model. Using VGG-16 we got 99.05%, 98.50%, and 99.69% on ArSL, ASL, and BSL respectively.},
 year = {2023}
}

Copy | Download

TY  - JOUR
T1  - Multiple Sign Language Identification Using Deep Learning Techniques
AU  - Ahmed Mahmoud Sultan
AU  - Waleed Makram Mohamed Zaki
AU  - Mohammed Kayed
AU  - Abdel Mgeid Amin Ali
Y1  - 2023/05/29
PY  - 2023
N1  - https://doi.org/10.11648/j.cssp.20231101.11
DO  - 10.11648/j.cssp.20231101.11
T2  - Science Journal of Circuits, Systems and Signal Processing
JF  - Science Journal of Circuits, Systems and Signal Processing
JO  - Science Journal of Circuits, Systems and Signal Processing
SP  - 1
EP  - 11
PB  - Science Publishing Group
SN  - 2326-9073
UR  - https://doi.org/10.11648/j.cssp.20231101.11
AB  - The research presents a general overview of sign languages, and a previous survey was conducted on all aspects of sign languages including the tools used to collect sign languages and the best algorithms to achieve the best results. A specialized database is prepared to combine the alphabet signs of the Arabic, American, and British languages, as they are the most important sign languages and the most widespread in the world. Based on different sign languages and deep learning techniques such as LeNet, VGG-16, and CapsNet, which are considered among the best methods for solving sign language problems based on our previous studies. The purpose of the research is to remove the communication gap between the deaf, and normal speaking people who speak one sign language or those who try to communicate from different countries and to identify these languages easily. We applied some of the traditional deep learning techniques such as LeNet, and then we applied VGG-16 using pre-training models and adjusted some layers to suit our problem. Also we applied CapsNet as it is perfectly suitable for solving the problem of sign language deformation, rotation, and scaling. The best results were achieved using VGG-16, as it was trained on a previous database like ImageNet, which contains millions of images. We got an accuracy of 99.69% when training the model of VGG-16, and an accuracy of 99.65% when testing the model. On the other hand, we got lower accuracies in CapsNet and LeNet compared to VGG-16. We got 96.54%, 97.45%, and 94.95% on BSL, ASL, and ArSL respectively while applying LeNet model, while we got 98.4848%, 98.4286%, and 99.5652% on ArSL, ASL, and BSL respectively while applying CapsNet model. Using VGG-16 we got 99.05%, 98.50%, and 99.69% on ArSL, ASL, and BSL respectively.
VL  - 11
IS  - 1
ER  -

Copy | Download