New image-driven model for hand gesture recognition developed

The research paper, published in the journal Cyborg and Bionic Systems, leverage a custom convolutional neural network (CNN) to extract both local and global features for classifying hand gestures, by decomposing high-density surface EMG (HD-sEMG) signals into channel-wise cumulative spike trains (cw-CSTs) and reconstructing these into two-dimensional images based on the spatial layout of the electrodes.

The research paper was prepared by scientists at Shanghai Jiao Tong University in China.

Gesture recognition, as a natural human-computer interaction method, has broad application prospects in fields such as prosthetic control, rehabilitation training, and mixed reality.

The commonly used surface electromyography (sEMG) signal analysis methods, such as those based on time-frequency domain features (such as RMS), often only capture rough neural control information, are susceptible to noise interference, and ignore the inherent spatial distribution characteristics of muscle movement.

“With the development of high-density surface electromyography (HD sEMG) technology, the discharge sequences (Spike Trains) of motor units (MUs) obtained through decomposition can more directly reflect the neural system's control over muscles, providing more representative low dimensional neural control information for gesture recognition.” said Yang Yu, a researcher at Shanghai Jiao Tong University.

The research process of this paper is divided into the following steps: Firstly, the HD-sEMG electrode array is used to collect the electrical signals of the forearm muscles, and the signals are filtered, denoised, and abnormal channels are removed.

Then, using an algorithm based on spatial propagation characteristics, the cumulative discharge sequence of each channel (cw-CST), is decomposed from the HD-sEMG signal to reflect the activity of adjacent motion units.

Next, the cw-CST data of each channel is reconstructed into a two-dimensional image (cw-CST image) based on the spatial distribution of electrodes to capture the spatial activation patterns of neural control.

Finally, a customised convolutional neural network is designed and trained for hand gesture recognition.

“Our study provides a novel and effective solution for high-precision gesture recognition, with the potential for widespread application in human-computer interaction fields such as prosthetic control and rehabilitation training.” said Yu.

—IANS