An Artificial Intelligence‐Based Motion Trajectory Prediction of Fibrous Matters

Understanding and predicting the motion behavior of fibrous matters is critical for studying the structure, performance, function, and assembly behavior of fibrous matters. However, due to the limitation of characterization technology on a space−time scale, the existing technologies cannot provide real‐time in situ tracking of entire dynamic processes of fibrous matters. To address this shortcoming, an artificial intelligence‐based codebase, named the artificial intelligence‐based fiber motion tracking (AI‐FMT), is proposed for analyzing, studying, and predicting motion behaviors of fibrous matters. The proposed AI‐FMT can automatically extract morphology information of fibrous matters from a large number of pictures or video streams, which serve as an input for machine learning to predict the motion pattern of fibrous matters. Using a finite‐element concept in understanding the point−coordinate relationship, the AI‐FMT greatly simplifies the architecture of the neural network during motion prediction. For the case of 928 trainable parameters, an average accuracy of 97.8% is achieved in predicting variations in morphological parameters, such as mean square radius of gyration, end‐to‐end distance, and curvature in the movement process of animal hair. The presented results can help to understand the structure, assembly, performance, and function of fibrous matters.

fibrous matters. For instance, the existing dynamic AFM technology cannot realize panoramic imaging of the assembling process from a single-strand protein chain into a functional body, and the current ultrahigh-speed camera technology is still unable to monitor the microscale and macroscale motions of fibrous matter simultaneously. An alternative approach is a quasidynamic observation [27,28] that is to conduct sampling and analysis at specific time points of the structural evolution process and then reconstruct the image information obtained at different times according to the chronological order to obtain the information on the quasidynamic structural change. The current quasidynamic monitoring process is not highly effective due to the tradeoff between the test time and sampling interval. The testing process is highly time-consuming, but it needs a short time interval to ensure high-accuracy analysis.
To overcome these difficulties, this study compiles a codebase that combines image recognition and machine learning to provide better capabilities for analyzing, studying, and predicting the motion behaviors of fibrous matters. With the help of computer vision, a computer can directly conduct the analysis of movements of fibrous matters based on the in situ observation data and then use a deep neural network to discover and predict the motion pattern, which cannot be observed easily by human eyes, as well as changes in intrinsic morphological parameters during the movement process. In recent years, deep learning has been widely used in image and spectrum recognition and classification, [29][30][31][32] and its application scope has also expanded beyond information technology. Increasing attention has been placed on the methodology of applying machine learning to material science research, [33][34][35][36] and emerging work proves that machine learning plays an essential role in materials science research. [37][38][39][40] For instance, deep neural networks have been widely used to identify compounds and structural features from spectral information. [41][42][43][44][45][46][47] Recently, they have even been used to explore the spatial structure of proteins [48][49][50] and to guide the design of mechanically and structurally optimized composite materials. [51][52][53][54][55][56] The machine learning techniques based on image recognition have played a promising role in analyzing the results from electron microscopies [57,58] and fluorescence lifetime imaging. [59][60][61] The application of artificial intelligence to the analysis of the motion pattern of fibrous matters will undoubtedly deepen the exploration of the motion pattern and physical mechanisms of fibrous matters.
More specifically, this study examines the deformation of animal hair in water as an example of the structural evolution of fibrous matters and builds a library of fibrous matter morphology analysis functions. The codebase can automatically extract morphology information of fibrous matters from a large number of images or video streams and present it in the form of a coordinate sequence, which allows direct calculation and analysis of the intrinsic morphological parameters of fibrous matters, such as length, curvature, and end-to-end distance. In addition, artificial intelligence is used to predict the motion pattern of a fibrous matter based on the coordinate sequence. The proposed algorithm is named the artificial intelligence-based fiber motion tracking (AI-FMT). A prominent feature of the proposed AI-FMT is that it uses the finite element concept to determine the pointÀ coordinate relationship, which greatly simplifies the neural network design while ensuring high prediction accuracy. For instance, the neural network built with only 928 trainable parameters can achieve an average prediction accuracy of 97.8% of variations in intrinsic morphological parameters, such as length, mean square radius of gyration (MSRG), end-to-end distance, and curvature in the movement process of animal hair. Although this study considers only animal hair movement as a research object, the proposed AI-FMT can predict the motion pattern of any fibrous matter and can directly output the dynamic changes in intrinsic morphological parameters during the movement process of the fibrous matter. The results presented in this work can deepen the understanding of the structure, aggregation, behavior, and function of fibrous matters, and the proposed AI-FMT is expected to play an active role in the fields of material science and life science.

Results and Discussion
As shown in Figure 1, the framework of the proposed AI-FMT mainly includes three tasks: 1) extraction of a fibrous matter from an image, which is named the motion tracking; 2) calculation of intrinsic morphological parameters of the fibrous matter, which www.advancedsciencenews.com www.advintellsyst.com is named the morphology calculation; and 3) AI prediction of a motion pattern of the fibrous matter, which is named the motion prediction. The detailed algorithms of these three tasks are given in Experimental Section.

Motion Tracking
Two steps are conducted to extract image information of a fibrous matter from an image or a video stream, which are to detect the object and to set the coordinate order to the points on the fibrous matter. Object detection is achieved by the operations of image reading and object enhancement, which are to distinguish a fibrous matter from the background, thus making the subsequent calculation more accurate. Setting the coordinate order refers to extracting and reorganizing the coordinates in order of their positions on the fibrous matter, which is consistent with human understanding. Setting the order to the coordinates is achieved by endpoint searching and trajectory connection. Next, a fibrous matter is recognized via adaptive binarization and maximum connected domain checking. Then, the program determines the two endpoints and connects them with an appropriate path. The path is represented by a coordinate sequence, as shown in Figure 2a. The correct coordinate sequence can be directly substituted into the morphological formula. If necessary, a new morphological calculation formula can be easily derived according to its definition. Another obvious advantage of using an ordered coordinate sequence to represent an image is the ability to establish a one-to-one relationship between different positions of points of interest at different time points during the observation. This is of particular importance for the automated analysis of points of interest. Using motion tracking, a large number of images or video streams can automatically be processed without the need for manual setting. In contrast, the commonly used ImageJ program can carry out a statistical calculation on only a limited amount of data under a manual setting. Although other fiber morphology analysis programs, such as FiberApp, [62] can conduct the fiber image analysis, their semiautomatic operation mode requires manual locating of a fibrous matter in every image frame during the observation process, which is not favorable to save the time of dynamic data analysis. In contrast, motion tracking achieved a success rate of 98.5% in reading a total of 86 059 frames of 25 video streams, providing the success rate of endpoint searching of 99.7% and the accuracy of trajectory connection of 98.9%, as shown in Figure 2d. b) The strategy of endpoint searching. The points whose mean values are greater than (n þ 1) 2 /4n 2 are excluded in every iteration by checking the mean value of the square region near the point to be judged, where n is the side length of the slicing window. c) Trajectory connection strategy. The contour is traversed clockwise, and the process meets the four-connection requirements. d) The success rate of the fiber motion tracking on the sample set. Recognition represents the success rate of endpoint searching. Success is the accuracy of trajectory connection. Accuracy is the accuracy of motion tracking, which represents the product of Recognition and Success.

Morphology Calculation
The morphological parameters of a fibrous matter, such as length, end-to-end distance, MSRG, and curvature distribution, usually reflect the movement mode, physical properties, and functions of the fibrous matter. For instance, for polymer chains, solution properties or viscoelasticity of flexible polymers can be determined by calculating the end-to-end distance and the MSRG. Once an image is converted into the coordinate sequence by motion tracking, morphological parameters of a fibrous matter, such as curvature of specific positions, can be easily calculated, as shown in Figure 3a,b and S1, Supporting Information. In addition to the global parameters' calculation, which is shown in Figure 3c, the coordinate sequence can also be used to locate the points of interest for partial analysis, as shown in Figure 3d. The specific scheme for calculating intrinsic morphological parameters is given in Supporting Information. In this work, 25 video streams are analyzed by the morphology calculation, and the length, MSRG, mean square of the end-to-end distance, average curvature, and other related parameters are calculated. By monitoring the changes in the morphological parameters during the movement process of a fibrous matter, the phenomenological movement pattern of the fibrous matter can be determined. When the morphological change of camel hair in water was used as a research object, the average curvature calculated by the morphology calculation gradually decreased, whereas the length remained basically the same, which indicated that the hydration deformation of camel hair followed the bending model rather than shrinking along the fiber's long axis. This conclusion is consistent with the result of the experimental observation. This result agreement also proves the accuracy of motion tracking in observing and analyzing the morphology of fibrous matters.

AI Prediction
Motion tracking can reduce data dimension by converting a fibrous matter image into a 2D coordinate sequence, which is of great help to machine learning. In a neural network, the tensor size of the input layer plays a decisive role in determining the number of network parameters and has a significant effect on the training speed. With the increase in the image size, a more complex neural network has to be used to determine data features, which requires a large number of training samples. For ultrahigh-resolution images, the number of pixels is in the order of millions. After being rarefied, only dozens of values are needed to store the shape of a fibrous matter, which greatly accelerates the training process. In this work, deformations extracted from videos are used to predict the morphological change of fibrous matters. The input is a short-term video stream of initial changes in the fibrous material or even a frame of the initial state. The output is the long-term deformation trend of a fibrous matter and a quantitative relationship between intrinsic morphological parameters and time.
When applying machine learning, it is difficult to learn the inputÀoutput pattern correctly if the training dataset is too small, which can also easily lead to the over-fitting phenomenon. The primary concern for machine learning is to find whether there is an attribute pattern (i.e., inherent law) of data of interest and whether the data dimension reduction can preserve the original attribute pattern of the data. Although converting an image into a coordinate sequence can greatly increase the training speed, the coordinate reference system may also result in misunderstanding of the data pattern. For instance, even when the hair shape does not change during rotation or a mirror-symmetric operation, a coordinate sequence representing the hair can significantly change. Therefore, when the obtained coordinate Figure 3. a) Image analysis conducted by motion tracking and morphology calculation modules. The 40 s video with a rate of 30 frames per second, which was converted into a coordinate sequence by the motion tracking module and then subjected to the overall or partial analysis of the morphology calculation module. In the overall analysis, the whole coordinate sequence is used to calculate the intrinsic morphological parameters, such as the length of a fibrous matter. In contrast, the partial analysis focuses on the properties of a point of interest, such as the curvature change. b) The calculation method of the curvature of a fibrous matter by the morphology calculation module. c) The average curvature of the whole object. d) The curvature of the point of interest.
www.advancedsciencenews.com www.advintellsyst.com sequence is flattened and directly fed to the input of a dense neural network, the image conversion process will represent a point-to-point coordinate conversion, which lacks the feature of generality. In this work, the pointÀcoordinate relationship is obtained based on the finite element concept and denoted as the finite eye sight (FES). By applying the method of divide and conquer, a many-to-one mapping is used instead of the one-to-one mapping. Specifically, the location of a point is represented by a short piece of coordinate sequence near the point, as shown in Figure 4a. In this way, the aforementioned problem can be significantly simplified. The pieces of coordinate sequences are relatively independent when being substituted into a dense neural network, and as a result, the deformation of a hair is decomposed into simple sections. Take the simplest three-to-one mapping as an example; in that case, not only object points will be considered but also the two closest points to the object point. The aim is to enable the neural network to perceive a longer segment when calculating one point. This operation is equivalent to adding more description information to the considered problem, which does not make the problem more specific but more general.
In this study, the deformation of camel hair in water was used as a research object, and machine learning was used to predict the hair movement process. The deformations of 25 camel hairs in water were recorded, and the corresponding coordinate sequences were extracted using motion tracking. The splitting rule of the overall data is given in Supporting Information. The input sequence was preprocessed with three-to-one mapping (slice ¼ 3). Only two hidden layers were used in the neural network. The mean square error (MSE) values of the training and validation sets were 73.9 and 49.7, respectively. Under the same conditions, the model based on the one-to-one mapping obtained the MSE of 52.7 and 122.9 on the training and validation sets, respectively. The significant decrease in the loss on the validation set has proven the good performance of the FES method. In the model with slice ¼ 3, two hidden layers and 77 training parameters were used, which indicated that the FES played a role in sparseness. When the slice length increased to five or seven, the loss did not decrease, indicating that the FES method did not require longer pieces of coordinates as an input, as shown in Figure 4c.

FES Neural Network Optimization
Although the FES achieved good results in training, the error was obvious. In the study of the deformation of camel hair, the shape changes are limited, namely, the equilibrium state will be reached in the final deformation process. Therefore, the closer the camel hair is to the final stable shape, the slower the shape changes and vice versa. If a neural network is too simple, it cannot detect the shape-changing speed accurately, so its prediction of the shape change will be either ahead or behind the actual value. Another problem that should be paid attention to is the potential imbalance in a dataset. When the camel hair approaches its stable shape, the shape change per unit time is very small, so in this stage, the shapes at different moments are highly similar to each other. In other words, compared with the samples in the unstable state, the samples in the stable state www.advancedsciencenews.com www.advintellsyst.com are less abundant, which leads to the imbalance in the training set.
To address the problem arising from a nonuniform shapechange speed, this article proposes to use a neural network to determine the deformation completion degree of the input coordinate sequence first and then assign the change weight and adjust the final change, as shown in Figure 5a and Figure S2, Supporting Information. For instance, when the object shape is close to the stable state, the neural network judges that the input coordinate sequence is in a high deformation completion state and assigns a small change weight, thus outputting the predicted shape with a small difference, as shown in Figure 5b. Figure 5. a) The architecture of the FES neural network. The input coordinate sequence was divided into multiple slices with different lengths. Then, the slices were transferred to the hidden layer to obtain an average change. The branch provided the change evaluation on the input coordinate sequence to obtain the change weight. The final output sequence was calculated according to the change weight. b) Optimization method. To determine the variable shape-change speed, it was necessary to evaluate the shape-change completion degree of the object. When the shape was in the stable state, the change evaluation was very small and even zero so as to limit the effect of the initial change. Consequently, the output coordinate sequence was very similar to the input one, which indicated that there was a small change in the stable state. c) Method of converting completion degree of deformation into a change weight. The output of the deformation completion degree was normalized by the rectified linear units (ReLU) activation layer. Then, the change weight was obtained by the evaluator, which was a linear mapping layer.  Specifically, images in the training set are graded, obtaining a value of one if an object is in the stable state and a value of zero if the object is in the unstable state. To achieve the earlier purpose, a branch neural network is added to the FES neural network to judge deformation completion degree. Next, the completion degree is linearly mapped to the weight value, as shown in Figure 5c, and the final coordinate sequence is adjusted according to this weight value. Inspired by the inception model, [63] different slice lengths are used to acquire the extended coordinate sequences to reduce the random error caused by the slice length. These sequences are then transmitted to the neural network in the corresponding shape. The results are averaged to obtain the primary change. Using this method, the FES neural network can examine the coordinate sequence with a different vision, and a random error of a single sequential model can be reduced. The loss values of the weighted result and the average primary result are shown in Figure 5d. The results show a smaller loss on the test set. Thus, the introduced modification was beneficial to dealing with a nonuniform shape-changing speed.
With the above-presented design and optimization, the FES neural network can predict the deformation of a fibrous matter with relatively high accuracy. After receiving the initial coordinate sequence, the FES neural network can provide the predicted shape of a fibrous matter at the next moment. In addition to the deformation prediction, the morphological calculation method can be applied to the prediction process to predict the morphological parameters of a fibrous matter directly. This function can be realized by the motion prediction module. This module can use the trained FES neural network to obtain predictions and then calculate the intrinsic morphological parameters for the sparse coordinate sequence obtained in an iteration. In motion prediction, the prediction can be conducted by discrete or continuous prediction. Discrete prediction uses the original data to predict the shape in the next moment, and continuous prediction uses the latest output to predict the shape in the next moment.
As shown in Figure 5e, four predictions were conducted on a sample in the test set at a 10 s interval. The image obtained by discrete prediction was almost identical to the original image and the image obtained by the motion tracking. In contrast, continuous prediction amplified the error values in the iterations. This was mainly because FES-based slicing of the coordinate sequence might not be in accordance with the overall evaluation of deformation. Camel hair was nonuniform in terms of both morphologies and deformation behavior. This situation caused different shape-changing speeds along the fiber axis. Here, the deformation degree of the whole fiber was evaluated, and the changing weight at different deformation stages was also adjusted. These operations made the forecast error of the overall shape-changing speed different from the local shape-changing rates for different fiber profile positions. In addition, due to the nonuniformity of deformation, deformation equilibrium might have been reached at some locations of the fiber, whereas other nearby locations might continue to deform. In this case, the equilibrium locations continued to deform (under the influence of the nonequilibrium locations) to allow the fiber to reach the final balance of the whole. This case is like the concept in machine learning that the results should achieve global optimization instead of local optimization. However, the overall deformation degree could not satisfy the coordination between different regions. Therefore, the wrong local shape-changing speed led to mismatched shape change and caused further enlargement of error. The MSRG calculated by the motion prediction module is shown in Figure 5f. In contrast to the images obtained by motion tracking, the MSRG result of the discrete prediction had an accuracy of more than 94.1%. In contrast, the final accuracy of the continuous prediction was 88.6%. The changes in the length, end-toend distance, and curvature were also analyzed, and an average accuracy of 97.8% was achieved, as shown in Table S1, Supporting Information. The obtained results have demonstrated that it is feasible to conduct deformation simulation and parameters' prediction of fibrous matters using multiple modules in the proposed AI-FMT.

Conclusion
In this study, a fiber morphology analysis method developed in Python is proposed for recording and predicting changes in the position of a fibrous matter under movement. The proposed method can provide fast and continuous analysis of a large number of fibrous matters. In addition, an FES neural network is trained to learn the changing rule of a fibrous matter, so the AI-based prediction of the fiber motion change is realized, for instance, the prediction accuracy of the MSRG when the animal hair shape change is 94.1%. The key to the good prediction performance of the proposed AI-FMT is that the developed FES neural network uses the concept of the finite element to determine the pointÀcoordinate relationship. The FES method can greatly simplify the neural network design while ensuring high prediction accuracy. The integration of the image-processing module, machine learning, and AI prediction technology provides the capability of accurate simulation and prediction of the changing trends of a fibrous matter's shape and intrinsic morphological parameters. The obtained results can be directly used for the analysis of the structure, assembly, performance, and function of the fibrous matter.
The proposed FES method can reduce the correlation of the overall motion of fibrous matters to a certain extent. Consequently, in the continuous prediction, inconsistency will cause outliers, thus increasing the error. Therefore, a recurrent neural network could be used in future research. Also, by learning the position relationship of neighboring points, it is feasible to constrain the predicted position so as to avoid excessive deviation. Thus, a loss function based on the deformation energy could be designed so as to eliminate the misleading effect of the coordinate sequence on the MSE.
Overall, the proposed method has achieved the goal of providing a high prediction accuracy at minimum cost. With the advantages of high prediction accuracy and wide application scope, the AI-FMT could provide practical guidance for future research in the interdisciplinary field, including material science and artificial intelligence.

Experimental Section
AI-FMT Compilation: The AI-FMT was developed using Python 3.8. Its operation required the support of the following modules: OpenCV-Python www.advancedsciencenews.com www.advintellsyst.com Camel Hair Deformation Recording: Twenty-five dry camel hairs were immersed in water, and the observed deformations were recorded. The diameters of camel hairs were about 30-40 μm. The length varied from 3 to 8 cm. To record the deformation of camel hairs, they were placed in a glass Petri dish filled with DI water at room temperature. The videos had a rate of 30 frames per second. The videos were segmented such that only one camel hair was shown in the field of view and then saved as images. The aspect ratios of images of different camel hairs were not uniform. The motion tracking module in the AI-FMT was used to extract coordinate sequences, which did not need to deal with the aspect ratio.
Camel Hair Deformation Records: For the coordinate sequence obtained by the motion tracking module, ten points were used to approximate the whole curve to reduce the amount of calculation, and the point-to-point relationship was established. The sampling interval was set to 10 s, which was long enough to detect the deformation between two adjacent samples. The training and validation sets were obtained from the data of the first 20 camel hairs, with a splitting ratio of 7:3. The time order and group classification of the samples were shuffled randomly. The test set was obtained from the data of the last five hairs, with the time order and group classification remaining intact.
FES Neural Network's Architecture and Training Process: The FES neural network was a multioutput structure compiled by functional API. It included two main tasks: change prediction and weight change evaluation. The network input was a coordinate sequence with a shape of (10, 2) obtained by motion tracking, which composed of ten sets of coordinates located on the fibrous matter. Then, the input was expanded with 2n coordinates nearby, so the shape became (10, 4n þ 2), where n ¼ 3, 5, 7. The missing coordinates were replaced with zero vectors. Each of the expanded coordinate sequences was processed by two hidden layers of the FES network to obtain the predicted coordinate sequence shape (10, 2), which was compared with the shape at the next moment. Then, the three output results were averaged to reduce the effect of the systematic error of different slice values. In the proposed model's results, the average result was denoted as Average. The input of the change evaluation was the same coordinate sequence as that of the change prediction. After processing the input data using four hidden layers, the completion degree of deformation was obtained. The parameters of the activation layer rectified linear units (ReLU) were as follows: alpha ¼ 1, max_value ¼ 1, threshold ¼ 0. The completion degree was linearly mapped to change weight c. The linear mapping layer was trainable as well. To obtain the output sequence, the primary change x' and input sequence x were added to the weight change, c, as follows where x þ λΔx represents the output sequence denoted as Weight in the proposed model.
The FES neural network was designed and trained using the machine learning library Keras with TensorFlow (2.3.1) backend in Python (3.8). The MSE was used as a loss function. The Adam was used as an optimizer. The network was trained for 3000 epochs using an NVIDIA Quadro RTX 5000. The whole training set was fed to the network in every epoch. The training time was between 20 and 25 ms per epoch except for the first epoch, whose training time was about 1.6 s. The simple sequential models that are shown in Figure 4b were compiled separately, but they were parts of the FES neural network. They shared the same training parameters with the FES neural network.
AI-FMT Morphological Parameters' Prediction Process: There are four steps to predict the change of morphological parameters. First, motion tracking was used to get the input coordinate sequence for FES neural network. The motion tracking calculated ten sets of coordinates located on the fibrous matter and converted them into an array with a shape of. (10,2) second; a predicted coordinate sequence was then obtained by FES neural network. Third, a profile curve of the fibrous matter was generated by AI-FMT according to the predicted ten sets of coordinates.
Fourth, the morphological parameters were calculated by Morphology Calculation.

Supporting Information
Supporting Information is available from the Wiley Online Library or from the author.