Artifacts Mitigation in Sensors for Spasticity Assessment

Spasticity is a pathological condition that can occur in people with neuromuscular disorders. Objective, repeatable metrics are needed for evaluation to provide appropriate treatment and to monitor patient condition. Herein, an instrumented bimodal glove with force and movement sensors for spasticity assessment is presented. To mitigate noise artifacts, machine learning techniques are used, specifically a multitask neural network, to calibrate the instrumented glove signals against the ground truth from sensors integrated in a robotic arm. The motorized robotic arm system offers adjustable resistance to simulate different levels of muscle stiffness in spasticity, and the sensors on the robot provide ground‐truth measurements of angular displacement and force applied during flexion and extension maneuvers. The robotic sensor measurements are used to train the instrumented glove data through multitask learning. After processing through the neural network, the Pearson correlation coefficients between the processed signals and the ground truth are above 0.92, demonstrating successful signal calibration and noise mitigation.


Introduction
About 75 million people worldwide are afflicted by neuromuscular disorders arising from stroke, cerebral palsy, spinal cord injury, multiple sclerosis, Parkinson's, etc. and live with a debilitating condition known as spasticity. [1] Spasticity results from damages to the regions of brain and spinal cord that control voluntary movements. It is a chronic but nonprogressive disorder that afflicts across all age groups. Spasticity is manifested as an abnormal muscle stiffness that interferes with movement, gait, and speech and causes great discomfort and pain. [2] A recent survey [3] of people with spasticity found that 72% reported lower quality of life, 44% reported loss of independence, 44% reported depression, 38% inability to work, etc. Untreated symptoms [4][5][6][7][8] can become permanently disabling, causing severe bone/ joint deformities and reduction in joint biomechanical range of motion that lead to patients in costly assisted living arrangements.
The evaluation of spasticity severity level is a critical step in selecting appropriate treatments for patients. To judge the severity of spasticity, clinicians perform standardized maneuvers, such as limb extension and flexion, as shown in Figure 1, to gauge the muscular resistance to movement. The current clinical benchmarks for assessing spasticity are the Modified Ashworth Scale (MAS) or Tardieu Scale, both of which are based on the clinician's subjective perception and limited in consistency and low sensitivity [9][10][11][12][13][14][15] that preclude fine-tuning of treatments. Researchers are developing different sensors and biomechanical tools to enable objective measurements on spastic muscles. Surface electromyography [16][17][18][19][20] was shown to capture involuntary muscle activations, but there are issues with environmental noise and signal reproducibility due to variations in electrode locations and skin conductivity. Biomechanical devices are used to quantify a muscle's resistance to being stretched. [21,22] Motorized systems are developed to measure the muscle response at specific maneuver speeds. [23][24][25] Multimodal measurements [26][27][28][29][30] to simultaneously record electromyography, movement, and muscle torque are shown to capture the velocity-dependent characteristics of spastic muscles. However, the prior approaches tend to place the instruments Spasticity is a pathological condition that can occur in people with neuromuscular disorders. Objective, repeatable metrics are needed for evaluation to provide appropriate treatment and to monitor patient condition. Herein, an instrumented bimodal glove with force and movement sensors for spasticity assessment is presented. To mitigate noise artifacts, machine learning techniques are used, specifically a multitask neural network, to calibrate the instrumented glove signals against the ground truth from sensors integrated in a robotic arm. The motorized robotic arm system offers adjustable resistance to simulate different levels of muscle stiffness in spasticity, and the sensors on the robot provide ground-truth measurements of angular displacement and force applied during flexion and extension maneuvers. The robotic sensor measurements are used to train the instrumented glove data through multitask learning. After processing through the neural network, the Pearson correlation coefficients between the processed signals and the ground truth are above 0.92, demonstrating successful signal calibration and noise mitigation. on patients, requiring frequent adjustments to fit the tools to different patient sizes.

FULL PAPER
To address the fitting issue, we developed an instrumented glove to be worn by a physician while conducted assessment maneuvers on patients. The glove only needs to fit the physician user, so that our tool can be readily applied to different patients. The instrumented glove record the force applied to move the spastic muscle F and the maneuver velocity v through flexible force-sensitive sensors [31][32][33][34] and an inertial motion unit (IMU), respectively. These two parameters are measured to determine the power P exerted to move the spastic muscle (P ¼ F * v), as a third parameter in spasticity evaluation.
In our previous work, the data from the sensors was compared with clinical MAS scores. [35] Our intention was to treat physicians' MAS ratings as the "ground truth." But in our study with two physicians and five individuals with cerebral palsy, there was only 27% agreement between the two physicians. To address the problem of "ground truth," we developed a mock patient with arm structure connected to a manually adjusted brake disk that was used to provide specific resistance. In this system, the "ground truth" power required to move patient arm and the power measured by the instrumented glove showed general agreement, but the Pearson correlation coefficient was only r ¼ 0.64. [35] We identified the short comings in this system including difficulties in setting desired resistance levels, multiple clock domains in the electronic instrumentation, and limitations in the linear signal processing algorithms to eliminate nonlinear artifacts.
To further improve our instrumented glove for spasticity assessment, this article presents an approach to improve sensor calibrations through supervised machine learning. This work includes the development of 1) a robotic mock patient offering the ability to dial in reproducible resistance, incorporated with sensors to log "ground truth" of the force applied and angular displacement of the maneuvers performed on the robot; and 2) a multitask neural network engine to mitigate sensing artifacts of the glove and calibrate the measurements for estimating physical parameters associated with spastic muscles. The machine learning approach enables to account for both sensor artifacts and other nonlinear effects such as grip variation, estimating accurate position from the accelerometer and gyroscope data from the IMU sensor, and so on. We validated the system with multiple users wearing the instrumented glove, performing flexion and extension movements on the robotic arm at multiple resistance levels, at multiple speeds, and on multiple days. These data were used to train and test the multitask neural network. The neural network removes artifacts in the sensor measurements. After passing the sensor signal through the network, we computed the Pearson correlation coefficients to compare the correlation between the processed signals and the ground truth, to quantify the effectiveness of this method.

Robotic Arm
Our instrumentation includes a new robotic "mock patient," which offers programmable resistance levels to simulate different severity of spasticity. The robotic mock patient is built out of laser-cut wood structures and stainless steel brackets. [36][37][38] The plastic arm is connected to a motor (Pittman, ID33003) through a gear train with a ratio of 1:15 to increase the motor torque. The motor of the robotic arm produces programmable resistance; namely, an increase in resistance requires the user to apply higher force to move the arm. To calibrate the resistance level  produced by the motor, we hung weights on the wrist of the robotic arm, and we applied current to the motor in the opposite direction and recorded the input needed to keep the arm fixed in the horizontal position. We calibrated the robotic resistance by applying 1-5 kg of mass, at 0.5 kg increments, to reach the resistance range commonly seen in spastic muscles. [23,39,40] The robotic arm has an optical encoder (USdigital, S1 optical shaft encoder) to monitor the angular position of the arm shaft, and a load cell (Transducer Techniques, ESP-10) to measure the force applied on the arm, as shown in Figure 1a,b. The load cell was calibrated by applying weights (1-5 kg) to the robotic arm when it was held at horizontal position. The optical encoder arrived with calibration already done by the manufacturer and was used as is. The optical encoder, the load cell, and the motor controller are connected to a controller board (National Instrument, Multi Function I/O Device USB-6211) to acquire the sensor data and control the motor. The custom user interface is developed using LabVIEW and sets the motor output with respect to the angular position of the robotic arm, during flexion and extension movements ( Figure 1c). The load cell and optical encoder on the robotic arm provide the ground-truth readings of the applied force and angular displacement, respectively, against which the instrumented glove sensors would be compared with.

Instrumented Glove
The instrumented glove [34,35] in Figure 1d has force sensors attached to the palmar side of the glove, and the sensors are based on force-sensitive resistors (Tekscan, 4256E). The force sensors have a total of 349 sensing elements in 18 sensing regions where every element outputs an 8-bit value (0-255) for a pressure range of 12 psi. As the sensor locations are known, the sensor readings provide a spatial map of applied force. The glove has also an inertial measurement unit (IMU) (motion node) with accelerometer, magnetometer, and gyroscope attached on the back side of the glove. The force sensors measured the force applied to move the robot arm, and the IMU captured the velocity of the movement.

Experimental Procedure
For each measurement trial, the user wore the instrumented glove and performed the flexion and extension maneuvers on the robotic arm of the mock patient system, as shown in Figure 1e,f, respectively. The programmable mock patient delivers a variable resistance as a function of position. The force from the robotic motor opposes the movement direction, and hence, adds resistance to the user's maneuvers. The robotic resistance is essentially the opposing force to the user's movement, as the resistance mass is converted to force by multiplying the resistance mass by the acceleration constant of gravity. The resistance of the robotic arm is independently adjustable for flexion and extension, as shown in Figure 2. The resistance is set to ramp up at position x 1 , to reach the desired level at position x 2 and then held constant to position x 3 , and ramp back down by position x 4 . The chart in Figure 2 shows the possible range for the input parameters to the robotic arm, and the default column indicates the settings we used for all the tests in this article. Table 1 shows the experimental conditions tested in various combinations. There were three individuals carrying the same tests over 5 days. On each day, 20 test cases were performed, by varying the robotic arm resistance by five levels, and with the users moving at four different speeds. We defined a single flexion or extension movement as one maneuver. For the different movement speeds, the user tried to follow the sound cue from a metronome to complete one maneuver per sound beat. There were also trials where no sound cue was provided, and the users were allowed to move at a random speed, but practically close to range of 40-60 maneuvers min À1 . Overall, we have collected a total of 300 test cases, with simultaneous recordings by the glove sensors and the robotic sensors, for the next step of developing a neural network model to reduce noise and artifacts in the instrumented glove system.

Supervised Machine-Learning Approach: Using Neural Network Model to Remove Artifacts
Artificial Neural Networks are a family of machine learning algorithms that are inspired by biological neurons in the human brain. The idea behind neural networks is to have networks of interconnected nodes (neurons) that learn to combine the input information in a manner that helps with the prediction of its corresponding task. There are different types of neural network layers. The ones used in this work are fully connected layers, [41]   as seen in a multilayer perceptron) and convolutional layers. [42] Convolutional layers [42] are more suitable to images as the layers learn convolutional kernels that extract useful spatial information (like edges or corners) from the image. Most neural networks attempt to solve individual problems, for example, just predicting the angular displacement using information from the IMU or predicting the resistance using force sensors attached to the glove. In this work, we attempt to use multitask learning, where we try to solve both these problems using the same network. The model will be more generalizable as compared to training on individual tasks by sharing information between these tasks.
To remove grip variations and other nonlinear artifacts affecting the measurements recorded by the glove, we trained neural network models specific to individual users. As both force and velocity measurements are taken during the same motion, and hence, are likely to influence each other, we adopt a multitask model to simultaneously process readings from the pressure sensors and the IMU on the glove. A multitask learning approach was taken as it has the following advantages: 1) Implicit data augmentation: multitask learning [43] effectively increases the sample size that we are using for training our model. As different tasks have different noise patterns, a model that learns two tasks simultaneously is able to learn a more general representation. Learning just task A bears the risk of overfitting to task A, whereas learning A and B jointly enables the model to obtain a better representation through averaging the noise patterns. 2) Attention focusing: if a task is very noisy or data are limited and high-dimensional, it can be difficult for a model to differentiate between relevant and irrelevant features. Multitask Learning [43] can help the model focus its attention on those features that actually matter as other tasks will provide additional evidence for the relevance or irrelevance of those features. 3) Regularization: multitask learning [43] models act as a regularizer by introducing inductive bias and reduce the risk of overfitting.

Data Preparation
The data collected by the instrumented glove had 725-dimensional feature vectors corresponding to measurements from the force sensors and 27 feature vectors corresponding to the data from the IMU. The data from the glove force sensors were represented as an image of size 29 Â 25 that shows the spatial correlation between individual force sensors. In this 29 Â 25 image, only a 16 Â 20 subsection (around the fingers) had usable information. We cropped out this 16 Â 20 block and feed it to the convolutional layers. The sensors around the fingers (16 Â 20 block) had the most contact with the robotic arm, whereas the remaining sensors on the hand did not always stay in contact with the arm depending on the grip used by the user. Therefore, we decided to only keep the region that was consistently in contact with the arm for purposes of prediction.

Model Design
The input force data had spatial characteristics based on sensor locations on the glove. We chose convolutional layers [42] to take advantage of this spatial correlation with its translational equivariance. Our data blocks do not belong to the same domain as the images present in ImageNet or other large datasets. Therefore, it would be improper to use pretrained weights from architectures trained on ImageNet. Moreover, our blocks are simpler in nature as compared with those in ImageNet, and using a very deep network could lead to overfitting. We used trial and error with different number of convolutional layers and chose the best configuration which showed low loss on both training and validation data for prediction of resistance and angular displacement. The entire network was built from scratch with no use of pretrained weights. Moreover, we tried both sigmoid and tanh activations, but the ReLU [44] activation unit showed lower loss during training and validation.
The convolutional encoder block [42] consists of two convolutional layers [42] with 128 filters with a kernel size of 3 Â 3, followed by a maxpooling layer, followed by two more convolution layers with each having 64 filters of size 3 Â 3, followed by global average pooling. [45] All the nonlinearities used in this subnetwork are rectified linear units (ReLU2). [44] When we pass the force sensor data block (of shape batch size Â 16 Â 20 Â 1) through the convolutional encoder block, we obtain a batch size Â 64-dimensional vector which represents the force readings. This vector is considered as a feature vector representation of the force sensors which will be combined with a similar representation of the IMU information to make predictions regarding resistance and angular displacement.
The data from the IMU are a batch-size Â 27-dimensional vector. As shown in Figure 3, the IMU data are passed through the dense fully connected layers (dense1 and dense2) to create a batch size Â 64-dimensional feature vector representation of the IMU (dense 2).
The vector representations of the IMU and the sensor readings are concatenated (concat in Figure 3) and fed into further fully connected layers. This is done to have a learned feature vector representation (dense4) that combines insights from both, the force sensors and the IMU. This learned representation is fed into two subnetworks, one which predicts resistance weight and the other that predicts the angular displacement in a multitask fashion, as shown in Figure 3, as the outputs.

Training the Neural Network Model
The model was built in python 3.6 using Tensorflow 1.12. The model currently uses 123 070 free parameters that can be trained. The process flow of the model development and results comparison is shown in Figure 4.
We trained the model with data for 4 days and tested it against data from the fifth day. For example, training was done with the data collected on days 2,3,4,5, and testing was done against data collected on day 1. We rotated through the entire dataset and carried out a fivefold cross validation to evaluate the models. The model was trained using a mean squared error loss and an Adam optimizer [46] with a learning rate of 0.00001.
We name the output of the neural network as "prediction" which are the resulting values after passing through the network to remove artifacts incurred during data acquisition. It is termed prediction, to emphasize that the test data was not included in the training for the neural network development. The prediction data will be compared with the ground-truth values from the sensors on the robotic arm, to determine the effectiveness of the neural network in tuning the glove signals to match the ground truth.
Finally, we calculated the power exerted to move the robotic arm by determining the product of force and velocity (P ¼ F * v). The predicted force was computed by scaling the predicted resistance by the gravity acceleration constant g ¼ 9.8 m s À2 . The predicted position is differentiated to compute angular velocity and filtered with a 5 Hz cutoff low-pass filter.
The filtered angular velocity is scaled by the robotic arm length (40 cm) to obtain the linear velocity.

Sensor Recordings of Flexion/Extension Maneuvers
The flexion and extension maneuvers as recorded by the sensors on the robotic arm and on the instrumented glove are shown in Figure 3. Architecture of the multitask learning neural network. The inputs are 1) spatial force map from the force sensors and 2) motion parameters from the IMU on the glove. A sequence of convolutional layers or dense layers extracts data from the sensors on the glove (region inside the orange box) and convert them into vector representation. The vector representations of the IMU and the force readings are concatenated and fed into further dense layers to yield outputs of resistance and angle. Figure 4. The top pane shows training the multitask neural network N and the bottom pane shows the use of N in calibrating the system. The labels F, M, P, and V correspond to force, motion, position, and velocity data, respectively. The subscripts m, g, and c refer to measured, ground truth, and corrected, respectively. The index k denotes multidimensional data; k ¼ 320 for the force sensor and k ¼ 27 for the IMU. The index n corresponds to time, sampled at 100 samples per second. 2000106 (5 of 9) Figure 5. There was a common data acquisition clock from the one control interface, with a sampling rate of 100 Hz for all sensors. The measurements showed periodic characteristics, where each period corresponded to a cycle of a flexion followed by an extension movement. In the robotic system, the motor produced a resistance depending on the input voltage (Figure 5a), and the load cell monitored the force applied to move the robotic arm (Figure 5b), whereas the optical encoder tracked the angular displacement as the robotic arm was rotated (Figure 5c). Simultaneously, the IMU records the angular velocity of the movement (Figure 5d), and the force sensors on the glove recorded the applied force (Figure 5e). The arbitrary unit is the result of summing the output from all the force sensor elements on the glove, and the arbitrary unit is converted to a resistance or force unit by the manufacturer calibration method (Tekscan) or by the neural network. Figure 5 shows a zoomed-in view of two cycles in a typical recording. The full recording for each trial usually lasted 20-30 s, as shown in Figure 6. Figure 6 shows an example set of data collected when the user moved at random speed to perform flexion and extension maneuvers on the robotic arm set at a resistance of 2.5 kg.

Neural Network Processing
The user was wearing the instrumented glove during the maneuvers, and thus, the force sensors on the glove tracked the applied force to overcome the resistance of the robotic arm, as shown in Figure 6a. There was general agreement between the glove force sensors (the orange line) and the robotic load cell measurements serving as the ground truth (the blue line). The recording by the glove sensors changed in sync with the ground truth. However, we also observe discrepancies between the glove measurements and the ground truth, and to compensate for these nonlinear artifacts, the glove measurements were fed into the neural network to yield a prediction output in Figure 6b. During the flexion and extension movements, the angular displacement of the robotic arm is monitored by the optical encoder serving as the ground truth. The IMU on the instrumented glove transverses the same angular displacement as the robotic arm. Therefore, we obtained velocity measurements from the gyroscope of the IMU and calculated the time integral of the angular velocity, to compute the angular displacement. In Figure 6c, the integration result (orange line) was poor and noisy, not acceptable for further analysis. Alternatively, we fed the IMU data into the neural network to arrive at the prediction of angular displacement in Figure 6d. The prediction from the neural network comes close to the ground-truth values. At each time instance, the neural network takes in feature vectors from the force sensors and the IMU on the glove and produces prediction values. In Figure 7a, each dot shows the value at each time instance, and the measured resistance as a function of position are mapped out. The resistance is in the direction opposite to the direction of motion. We defined the resistance against extension to be positive in sign, and the reverse (flexion) to be negative. The neural network predictions closely follow the ground truth.
To quantify the correlation of the power between the prediction and the ground truth, we computed the Pearson correlation coefficients, as shown in Figure 7b. For this set of data, there were 20 trials for each day done by the same user. Within each day, the robotic resistance was set to vary between 1 and 3 kg, whereas the user performed the maneuvers at four different speeds, as shown in Table 1. The different movement speeds widened the spread in the correlation coefficients, but within our chosen speed range there is no obvious difference between Figure 6. An example set of data collected at a resistance of 2.5 kg as the user moved at random speed. a) The resistance measured by the glove (orange), versus the resistance as measured by the load cell that serves as ground truth (blue). b) After neural network processing, the predicted resistance (orange) compared with the ground truth (blue). c) The angular velocity measured by the IMU gyroscope and integrated over time to yield angular displacement (orange), versus the angular displacement as measured by the optical encoder that serves as the ground truth (blue). d) After neural network processing, the predicted values (orange) compared with the ground truth (blue). Each time frame is 10 ms.  maneuvers done following the metronome cue or without the cue. After neural network processing to remove artifacts, the prediction power was highly correlated to the ground truth, with average Pearson correlation coefficient above 0.91 for one user. Furthermore, we summarized the Pearson correlation coefficients in Figure 8 to compare the outcomes of applying individualized neural networks for three different users, running trials across 5 days. The networks were able to remove artifacts in the resistance and angular displacement recordings by the instrumented glove, as evident in the high correlation >0.9 between these parameters and the ground truth values. Remarkably, the angular displacement predictions reached 0.99 in correlation for all three users. The power exerted to move the robotic arm was determined from the product of the resistance force and angular velocity (Figure 4), and the power values were also shown to be well correlated >0.92 between the neural network prediction results and the ground truth.

Discussions
A successful treatment for a patient that is suffering from muscle spasticity can only be achieved if there is an objective monitoring of the disease. Subjective and inconsistent assessment of spasticity has major implications on patient care, because accurate assessment is needed to determine the dosage of medication the patient receives and affect the therapy outcome.
Thus, this work aims to improve the tool for the assessment of spasticity severity.
Our instrumented glove previously was calibrated without using the neural networks, and the Pearson correlation coefficient between the glove measurement and the ground truth was only 0.64. [35] The improvements in our current system originated from better artifacts removal by the neural network, compensating for the nonlinear background contributions such as user grip variations, movement speed, etc. We showed that the supervised machine-learning technique presented here can be applied to multiple types of sensors ranging from force to movement sensors.
The robotic system also facilitated improved data collection, as the mock-patient arm resistance were easily changeable by the programmable motor and lessened the time and labor involved in tuning the calibration settings. Hence, we were able to collect hundreds of trials in a week to provide enough training data for the neural network implementation. More data collection to increase the pool of users will be carried out in the future, to potentially create a neural network model that can be generalized across multiple users.

Conclusions
As a biomechanical tool for monitoring spasticity, the instrumented glove demands better calibration approaches to remove artifacts in the sensor measurements. Here, we used supervised machine learning enabled by the development of a robotic arm system, to demonstrate improvements in signal quality. The robotic arm system is designed to offer programmable resistance and is integrated with sensors that serve as the ground truths for calibrating the instrumented glove. Through implementing multitask learning neural networks, the processed signals of the glove sensors showed high correlation to the ground-truth values. The parameters of resistance, angular displacement, and power were analyzed, and the Pearson correlation coefficients between the processed signals and the ground truth are above 0.92, demonstrating successful signal calibration and noise mitigation. This instrumented glove with the improved calibration will be a better recording tool for future assessments of spasticity in patients.