Publications

2024

1. 

   CNN-LSTM and post-processing for EMG-based hand gesture recognition

   Lorena Isabel Barona López , Francis M. Ferri , Jonathan Zea, and 2 more authors

   Intelligent Systems with Applications, 2024

   Abs Bib PDF

   

   Hand Gesture Recognition (HGR) using electromyography (EMG) signals is a challenging problem due to the variability and noise in the signals across individuals. This study addresses this challenge by examining the effect of incorporating a post-processing algorithm, which filters the sequence of predictions and removes spurious labels, on the performance of a HGR model based on spectrograms and Convolutional Neural Networks (CNN). The study also compares CNN vs CNN-LSTM to assess the influence of the memory cells on the model. The EMG-EPN-612 dataset, which contains measurements of EMG signals for 5 hand gestures from 612 subjects, was used for training and testing. The results showed that the post-processing algorithm increased the recognition accuracy by 41.86% for the CNN model and 24.77% for the CNN-LSTM model. The inclusion of the memory cells increased accuracy by 3.29%, but at the cost of 53 times more learnables. The CNN-LSTM model with post-processing achieved a mean recognition accuracy of 90.55% (SD=9.45%). These findings suggest new paths for research in HGR architectures beyond the traditional focus on the classification and feature extraction stages. For reproducibility purposes, we made publicly available the source code in Github.

   @article{BARONA-LOPEZ-cnnlstm,
     title = {CNN-LSTM and post-processing for EMG-based hand gesture recognition},
     journal = {Intelligent Systems with Applications},
     volume = {22},
     year = {2024},
     issn = {2667-3053},
     doi = {https://doi.org/10.1016/j.iswa.2024.200352},
     url = {https://www.sciencedirect.com/science/article/pii/S2667305324000280},
     author = {{Barona López}, Lorena Isabel and Ferri, Francis M. and Zea, Jonathan and {Valdivieso Caraguay}, Ángel Leonardo and Benalcázar, Marco E.},
     keywords = {Hand gesture recognition, EMG-EPN-612, Post-processing, Convolutional neural networks, Long short-term memory, EMG, Spectrogram},
     google_scholar_id = {eQOLeE2rZwMC},
     dimensions = {true},
   }

2023

1. 

   Effects on Hand Gesture Recognition Accuracy after Severe Cranial Trauma

   Jonathan A. Zea, Luis Gabriel Macias Santillán , Lorena Isabel Barona López , and 2 more authors

   In 2023 IEEE Seventh Ecuador Technical Chapters Meeting (ETCM) , 2023

   Bib

   

   @inproceedings{zea2023effects,
     author = {Zea, Jonathan A. and Santillán, Luis Gabriel Macias and López, Lorena Isabel Barona and Caraguay, Ángel Leonardo Valdivieso and Benalcázar, Marco E.},
     booktitle = {2023 IEEE Seventh Ecuador Technical Chapters Meeting (ETCM)},
     title = {Effects on Hand Gesture Recognition Accuracy after Severe Cranial Trauma},
     year = {2023},
     volume = {},
     number = {},
     pages = {1-5},
     keywords = {Training;Adaptation models;Sociology;Gesture recognition;Muscles;Electromyography;Data models;HGR;EMG;cranial trauma},
     doi = {10.1109/ETCM58927.2023.10309078},
     google_scholar_id = {W7OEmFMy1HYC},
     dimensions = {true},
   }

2021

1. 

   An Open-Source Data Acquisition and Manual Segmentation System for Hand Gesture Recognition based on EMG

   Jonathan Zea, Marco E. Benalcázar, Lorena Isabel Barona Lôpez , and 1 more author

   In 2021 IEEE Fifth Ecuador Technical Chapters Meeting (ETCM) , 2021

   Bib

   

   @inproceedings{zea2021opensource,
     author = {Zea, Jonathan and Benalcázar, Marco E. and Barona Lôpez, Lorena Isabel and Valdivieso Caraguay, Ángel Leonardo},
     booktitle = {2021 IEEE Fifth Ecuador Technical Chapters Meeting (ETCM)},
     title = {An Open-Source Data Acquisition and Manual Segmentation System for Hand Gesture Recognition based on EMG},
     year = {2021},
     volume = {},
     number = {},
     pages = {1-6},
     keywords = {Training;Data acquisition;Gesture recognition;Manuals;Standardization;Electromyography;Physiology;HGR;Hand Gesture Recognition;Myo Armband;gForce Pro;Matlab;EMG},
     doi = {10.1109/ETCM53643.2021.9590811},
     google_scholar_id = {Y0pCki6q_DkC},
     dimensions = {true},
   }

2. ICI2ST

   “Spanish Políglota”: an automatic Speech Recognition system based on HMM

   Jonathan A. Zea, and Josafá Aguiar

   In 2021 Second International Conference on Information Systems and Software Technologies (ICI2ST) , 2021

   Bib

   

   @inproceedings{zea2021spanish,
     author = {Zea, Jonathan A. and Aguiar, Josafá},
     booktitle = {2021 Second International Conference on Information Systems and Software Technologies (ICI2ST)},
     title = {“Spanish Políglota”: an automatic Speech Recognition system based on HMM},
     year = {2021},
     volume = {},
     number = {},
     pages = {18-24},
     keywords = {Training;Maximum likelihood estimation;Synthesizers;Hidden Markov models;Training data;Audio recording;Acoustics;Automatic Speech Recognition;ASR;HMM;HTK;Julius;Voxforge;Spanish;Grapheme-to-Phoneme Conversion},
     doi = {10.1109/ICI2ST51859.2021.00011},
     google_scholar_id = {zYLM7Y9cAGgC},
     dimensions = {true},
   }

3. CSEI

   Feature Evaluation of EMG Signals for Hand Gesture Recognition Based on Mutual Information, Fuzzy Entropy and RES Index

   Lorena Isabel Barona-Lopez , Angel Leonardo Valdivieso-Caraguay , Marco E. Benalcazar, and 2 more authors

   In Advances and Applications in Computer Science, Electronics and Industrial Engineering , 2021

   Abs Bib

   

   In the field of hand gesture recognition based on electromyography,Barona-Lopez, the selection of features is still a matter of research because the accuracy of feature evaluation techniques depends on the distribution underling a feature space.Currently, there are a lot of kinds of features,so a selection method is required in order to find an optimal feature subset.The selection of features can be done based on either classifier methods or statistical measurement indexes. On one hand, the use of classifier methods takes considerable time, consequently, they are computationally expensive. On the other hand, the use of a statistical index allows to select features based on properties, and they are less expensive in terms of time and computational costs. In this context, this study analyzed different statistical indexes to discover optimal feature subsets and predicts which one may have the best performance in terms of classification and recognition accuracy. For this purpose, four indexes were analyzed: the sum of RES (separation and compactness index), the sum of Fuzzy Entropies, and both of these indexes divided by the mutual information of feature subset. These index values were calculated for 10 different feature subsets obtained from nineteen time-domain features which have been investigated by many researchers for extracting useful information from an electromyography signal. The feature subsets were implemented in four classifiers (Artificial Neural Network, k-nearest Neighbors Algorithm, Support Vector Machine, and decision trees) to find the best statistical index which tries to quantify the suitableness of the feature space. Each tested index in this work was analyzed in terms of classification and recognition accuracy for each machine learning algorithm.

   @inproceedings{barona2021feature,
     author = {Barona-Lopez, Lorena Isabel and Valdivieso-Caraguay, Angel Leonardo and Benalcazar, Marco E. and Aguas, Xavier and Zea, Jonathan A.},
     editor = {Garc{\'i}a, Marcelo V. and Fern{\'a}ndez-Pe{\~{n}}a, F{\'e}lix and Gord{\'o}n-Gallegos, Carlos},
     title = {Feature Evaluation of EMG Signals for Hand Gesture Recognition Based on Mutual Information, Fuzzy Entropy and RES Index},
     booktitle = {Advances and Applications in Computer Science, Electronics and Industrial Engineering},
     year = {2021},
     publisher = {Springer Singapore},
     address = {Singapore},
     pages = {101--119},
     isbn = {978-981-33-4565-2},
     doi = {https://doi.org/10.1007/978-981-33-4565-2_7},
     google_scholar_id = {UeHWp8X0CEIC},
     dimensions = {true},
   }

2020

1. MDPI

   An Energy-Based Method for Orientation Correction of EMG Bracelet Sensors in Hand Gesture Recognition Systems

   Lorena Isabel Barona López , Ángel Leonardo Valdivieso Caraguay , Victor H. Vimos , and 4 more authors

   Sensors, 2020

   Abs Bib

   

   Hand gesture recognition (HGR) systems using electromyography (EMG) bracelet-type sensors are currently largely used over other HGR technologies. However, bracelets are susceptible to electrode rotation, causing a decrease in HGR performance. In this work, HGR systems with an algorithm for orientation correction are proposed. The proposed orientation correction method is based on the computation of the maximum energy channel using a synchronization gesture. Then, the channels of the EMG are rearranged in a new sequence which starts with the maximum energy channel. This new sequence of channels is used for both training and testing. After the EMG channels are rearranged, this signal passes through the following stages: pre-processing, feature extraction, classification, and post-processing. We implemented user-specific and user-general HGR models based on a common architecture which is robust to rotations of the EMG bracelet. Four experiments were performed, taking into account two different metrics which are the classification and recognition accuracy for both models implemented in this work, where each model was evaluated with and without rotation of the bracelet. The classification accuracy measures how well a model predicted which gesture is contained somewhere in a given EMG, whereas recognition accuracy measures how well a model predicted when it occurred, how long it lasted, and which gesture is contained in a given EMG. The results of the experiments (without and with orientation correction) executed show an increase in performance from 44.5% to 81.2% for classification and from 43.3% to 81.3% for recognition in user-general models, while in user-specific models, the results show an increase in performance from 39.8% to 94.9% for classification and from 38.8% to 94.2% for recognition. The results obtained in this work evidence that the proposed method for orientation correction makes the performance of an HGR robust to rotations of the EMG bracelet.

   @article{lopez2020energy,
     author = {Barona López, Lorena Isabel and Valdivieso Caraguay, Ángel Leonardo and Vimos, Victor H. and Zea, Jonathan A. and Vásconez, Juan P. and Álvarez, Marcelo and Benalcázar, Marco E.},
     title = {An Energy-Based Method for Orientation Correction of EMG Bracelet Sensors in Hand Gesture Recognition Systems},
     journal = {Sensors},
     volume = {20},
     year = {2020},
     number = {21},
     article-number = {6327},
     url = {https://www.mdpi.com/1424-8220/20/21/6327},
     pubmedid = {33171967},
     issn = {1424-8220},
     doi = {10.3390/s20216327},
     google_scholar_id = {qjMakFHDy7sC},
     dimensions = {true},
   }

2. 

   Sign Language Recognition Using Wearable Electronics: Implementing k-Nearest Neighbors with Dynamic Time Warping and Convolutional Neural Network Algorithms

   Giovanni Saggio , Pietro Cavallo , Mariachiara Ricci , and 3 more authors

   Sensors, 2020

   Abs Bib PDF

   

   We propose a sign language recognition system based on wearable electronics and two different classification algorithms. The wearable electronics were made of a sensory glove and inertial measurement units to gather fingers, wrist, and arm/forearm movements. The classifiers were k-Nearest Neighbors with Dynamic Time Warping (that is a non-parametric method) and Convolutional Neural Networks (that is a parametric method). Ten sign-words were considered from the Italian Sign Language: cose, grazie, maestra, together with words with international meaning such as google, internet, jogging, pizza, television, twitter, and ciao. The signs were repeated one-hundred times each by seven people, five male and two females, aged 29–54 y ± 10.34 (SD). The adopted classifiers performed with an accuracy of 96.6% ± 3.4 (SD) for the k-Nearest Neighbors plus the Dynamic Time Warping and of 98.0% ± 2.0 (SD) for the Convolutional Neural Networks. Our system was made of wearable electronics among the most complete ones, and the classifiers top performed in comparison with other relevant works reported in the literature.

   @article{saggio2020sign,
     author = {Saggio, Giovanni and Cavallo, Pietro and Ricci, Mariachiara and Errico, Vito and Zea, Jonathan and Benalcázar, Marco E.},
     title = {Sign Language Recognition Using Wearable Electronics: Implementing k-Nearest Neighbors with Dynamic Time Warping and Convolutional Neural Network Algorithms},
     journal = {Sensors},
     volume = {20},
     year = {2020},
     number = {14},
     article-number = {3879},
     url = {https://www.mdpi.com/1424-8220/20/14/3879},
     pubmedid = {32664586},
     issn = {1424-8220},
     doi = {10.3390/s20143879},
     google_scholar_id = {2osOgNQ5qMEC},
     dimensions = {true},
   }

3. CSEI

   Real-Time Hand Gesture Recognition: A Long Short-Term Memory Approach with Electromyography

   Jonathan A. Zea, and Marco E. Benalcázar

   In Advances and Applications in Computer Science, Electronics and Industrial Engineering , 2020

   Abs Bib

   

   Hand gestures are a non-verbal type of communication ideally suited for Human-Machine Interaction. Nevertheless, accuracy rates and response times still are a matter of research. One unattended problem has been the difficulty and vagueness of the evaluation of the models proposed in the literature. In this paper, a protocol for evaluating recognition is proposed. A Hand Gesture Recognition system using electromyography signals (EMG) is also presented. This model works in real time, is user dependent and is based in Long Short-Term Memory Networks. The model recognizes 5 different classes (wave in, wave out, fist, open, pinch) apart from the relax state. A data set with 120 people was collected using the commercial device Myo Armband. The data set was divided 50% for tuning and 50% for testing. Following the evaluation protocol proposed, the presented model achieves a 95.79% in classification and a 88.1% in recognition accuracy. An analysis of the characteristics of this model shows the advantage over similar models and its capability for being applied in all sort of fields.

   @inproceedings{zea2019realtimeLSTM,
     author = {Zea, Jonathan A. and Benalc{\'a}zar, Marco E.},
     editor = {Nummenmaa, Jyrki and P{\'e}rez-Gonz{\'a}lez, Federico and Domenech-Lega, Bruno and Vaunat, Jean and Oscar Fern{\'a}ndez-Pe{\~{n}}a, F{\'e}lix},
     title = {Real-Time Hand Gesture Recognition: A Long Short-Term Memory Approach with Electromyography},
     booktitle = {Advances and Applications in Computer Science, Electronics and Industrial Engineering},
     year = {2020},
     publisher = {Springer International Publishing},
     address = {Cham},
     pages = {155--167},
     isbn = {978-3-030-33614-1},
     doi = {https://doi.org/10.1007/978-3-030-33614-1_11},
     google_scholar_id = {d1gkVwhDpl0C},
     dimensions = {true},
   }

2018

1. EUSIPCO

   Real-Time Hand Gesture Recognition Based on Artificial Feed-Forward Neural Networks and EMG

   Marco E. Benalcázar, Carlos E. Anchundia , Jonathan A. Zea, and 3 more authors

   In 2018 26th European Signal Processing Conference (EUSIPCO) , 2018

   Bib

   

   @inproceedings{benalcazar2018realtimeFFNN,
     author = {Benalcázar, Marco E. and Anchundia, Carlos E. and Zea, Jonathan A. and Zambrano, Patricio and Jaramillo, Andrés G. and Segura, Marco},
     booktitle = {2018 26th European Signal Processing Conference (EUSIPCO)},
     title = {Real-Time Hand Gesture Recognition Based on Artificial Feed-Forward Neural Networks and EMG},
     year = {2018},
     volume = {},
     number = {},
     pages = {1492-1496},
     keywords = {Electromyography;Computational modeling;Gesture recognition;Feature extraction;Solid modeling;Robot sensing systems;Hidden Markov models;hand gesture recognition;real-time;feed-forward neural networks;electromyography;feature extraction;time series},
     doi = {10.23919/EUSIPCO.2018.8553126},
     google_scholar_id = {u5HHmVD_uO8C},
     dimensions = {true},
   }

2017

1. 

   Real-time hand gesture recognition using the Myo armband and muscle activity detection

   Marco E. Benalcázar, Cristhian Motoche , Jonathan A. Zea, and 6 more authors

   In 2017 IEEE Second Ecuador Technical Chapters Meeting (ETCM) , 2017

   Bib

   

   @inproceedings{benalcazar2017realtime,
     author = {Benalcázar, Marco E. and Motoche, Cristhian and Zea, Jonathan A. and Jaramillo, Andrés G. and Anchundia, Carlos E. and Zambrano, Patricio and Segura, Marco and Benalcázar Palacios, Freddy and Pérez, María},
     booktitle = {2017 IEEE Second Ecuador Technical Chapters Meeting (ETCM)},
     title = {Real-time hand gesture recognition using the Myo armband and muscle activity detection},
     year = {2017},
     volume = {},
     number = {},
     pages = {1-6},
     keywords = {Electromyography;Muscles;Feature extraction;Gesture recognition;Optical fiber sensors;Real-time systems;hand gesture recognition;EMG;real time;myo armband;muscle activity detection;machine learning},
     doi = {10.1109/ETCM.2017.8247458},
     google_scholar_id = {u-x6o8ySG0sC},
     dimensions = {true},
   }

2. EUSIPCO

   Hand gesture recognition using machine learning and the Myo armband

   Marco E. Benalcázar, Andrés G. Jaramillo , Jonathan Zea, and 2 more authors

   In 2017 25th European Signal Processing Conference (EUSIPCO) , 2017

   Bib

   

   @inproceedings{benalcazar2017hand,
     author = {Benalcázar, Marco E. and Jaramillo, Andrés G. and Zea, Jonathan and Páez, Andrés and Andaluz, Víctor Hugo},
     booktitle = {2017 25th European Signal Processing Conference (EUSIPCO)},
     title = {Hand gesture recognition using machine learning and the Myo armband},
     year = {2017},
     volume = {},
     number = {},
     pages = {1040-1044},
     keywords = {Electromyography;Muscles;Gesture recognition;Real-time systems;Feature extraction;Hidden Markov models;Heuristic algorithms;Hand gesture recogntion;EMG;machine learning;k-nearest neighbor;dynamic time warping algorithm},
     doi = {10.23919/EUSIPCO.2017.8081366},
     url = {https://ieeexplore.ieee.org/abstract/document/8081366},
     google_scholar_id = {9yKSN-GCB0IC},
     dimensions = {true},
   }