Edge‐based effective active appearance model for real‐time wrinkle detection

Abstract Background Recently, the field of face and facial features has been progressively studied. The features of facial expression have gained increasing attention for related applications. The wrinkle is the most representative feature, and its research and applications have been topics of high interest. Wrinkles play an important role in face feature analysis. They have been widely used in applications, such as age estimation, skin texture classification, expression recognition, and simulation. Purpose Existing approaches to the image‐based analysis of wrinkles as texture not as curvilinear discontinuity and wrinkle detection mainly have focused on detecting wrinkles on forehead position, which is usually horizontal linear shapes, while the detection of the nasolabial wrinkle is not well understood due to their variety of shapes and complexity. Method In this paper, we present a nasolabial wrinkle line detecting effective algorithm based on the Active appearance model and Hessian filter to improve localization results by creating unique initial shapes of the wrinkle lines for each input face image. Results Experimental results show that the proposed method is capable of tracking curve wrinkle lines, thus allowing to detect complexly structured wrinkle lines. This work demonstrates results illustrated the competitiveness of the proposed method in detecting nasolabial wrinkle lines. Conclusion In our study, this was introduced the effectiveness of changing the structure of AAM and successfully applied in wrinkle line localizing, although competitive results are achieved by the proposed wrinkle detection method.


| INTRODUC TI ON
The detection of wrinkle positions is critical for applications like facial beauty within which the positions of wrinkle lines and regions must be detected before they are removed or trimmed. 1,2 The quantitative evaluation of skin condition has been an area of quite an intense study.
There is great interest in complementing the dermatologist's diagnostic visual assessment of skin with objective measures. These methods are also valuable for the efficient development of effective treatments. 3 Facial skin wrinkles are not only important features in terms of facial aging and beauty but also can also provide cues to a person's lifestyle and health condition. For example, facial wrinkles can recognize facial expression, 4,5 or whether the person has been a smoker. 6,7 Some of the factors influencing facial wrinkles are a person's lifestyle, genetic inheritance, ethnicity, overall health, skincare routines, and gender. Hence, computer-based analysis of facial wrinkles has great potential to exploit this underlying information for relevant applications. However, wrinkle regions continue to be manually located in various works, which could largely restrict their applicability. These wrinkle detection methods are limited to some simple wrinkles, and they are not enough accurate or general. To improve the efficiency of these methods, wrinkle detection should be automatic and general.
However, each method has its own strengths and weaknesses.
Current wrinkle detection methods focused on detecting active wrinkle positions as forehead wrinkles, [8][9][10] but detecting passive wrinkle position as cheek wrinkles is not effective with previous methods. The cheek wrinkles are more complicated and challenging. The above algorithms rely on a distinct boundary texture of the wrinkle region, and the information of texture around the wrinkle area is not sufficiently complete or effective for curve type wrinkle line detection.
To make nasolabial wrinkle detection method more efficient, this work is divided into two main parts: 1. Detecting initial position of wrinkle line by Hessian filter: To make wrinkle detection method more effective and common was detected candidate wrinkle position by applying a ridge detection algorithm.
2. Apply initial wrinkle line to Active Appearance Model as individual mean shape: Create an active shape model and localize the wrinkle line using initial shape of wrinkle line.
The main contribution of this work is summarized as follows: The information about related works on the wrinkle area was introduced in section 2. The proposed wrinkle detection method demonstrated in Section 3. Then, the experimental results and the corresponding figures are presented in Section 4. Finally, discussion and conclusion are in Section 5.

| REL ATED WORK
In this section, we introduce related work based on wrinkle line detection. We divided the section into three parts rely on methods that were used in the wrinkle research area: texture based, filters based, and shape model-based.

| Texture based
The automated location of wrinkles from images is an important step in age estimation. Ng et al 11 studied a different wrinkle region extractor for age estimation. This method works by applied a Canny operator to detect the wrinkles and represented it as a pattern of age estimation. The edge detector detects the boundaries of the pattern, and it could not be suitable for wrinkle localization. Batool and Chellappa 12 proposed to detect wrinkles by marked point processes, and the Markov chain Monte Carlo method was used to detect the initial positions of wrinkles. In Ref.,1 the authors proposed a method to detect forehead wrinkles, using a curve pattern as a soft biometric. The permanent wrinkles are often relatively simple with distinct shapes. Detection methods aiming at this type of wrinkle might not be useful for temporary wrinkles with nonlinear and blurry shapes.

| Shape model-based
Facial shape detection has received a lot of attention over the past decade and successfully applied in many research areas and have been topics of high interest using shape models to detect wrinkle line. The Active Shape Model (ASM) was used by training and locating 81 face feature points, including several points in the nasolabial region. 15 In addition, wrinkles in some previously extracted fixed areas were detected using geometric elements such as a change in Current algorithms leave many opportunities to improve wrinkle detection method and introduced by the following drawbacks: • Current wrinkle detectors rely on a bold boundary pattern of the wrinkle line.
• Texture and geometrical information around the wrinkle line not used completely.
• Above methods mainly suitable for detecting forehead wrinkle lines, wherein nasolabial wrinkle lines remain unexplored.

| PROP OS ED ME THOD
The proposed detector Figure 1 largely consists of three steps: first, detecting wrinkle position area; second, the ridge detection method is applied; and third, creating a unique initial shape and finally apply AAM using created initial shape.

| Wrinkle edge detection
The first and most important step is to detect face and face feature points from the input image. In this step, using general AAM where H xx , H xy , and H yy are the second derivative.  where S represents the eigenvectors of the shape, and x is the mean shape of Equation (3.8). By calculating this calculation, we take the mean shape of the database.

| Fitting active appearance model
The AAM requires an initial estimate to the location of the shape, the better is this estimate, and minor is a risk of being trap in a local minimum. The disadvantage of AAM using for every input image only one created an initial mean shape of the trained database that brings failings in detecting various shape position of wrinkle line.
Thus, Hessian filter was applied to create a unique wrinkle initial line for each input image, by wrinkle structures constructed to utilize the local deformation for shape variance modeling Table 1 where E represents the Eigen vectors of shape, and msh represents new unique mean shape.
To apply the AAM algorithm for sets of points with unique initial shape under Procrustes transformation and reduce the error between input image and model using the relationship between the shape and texture model was applied:l.
where W is a vectorization of wrinkle structure after Procrustes transformation, s is the spars coefficient corresponding to the wrinkle structure database, vec is the vectorization of the input shape, and is the regularization parameter 1e −6 .

| E XPERIMENTAL RE SULTS
This research investigates the effectiveness of detecting nasolabial wrinkle lines based on AAM using a unique initial shape. The proposed algorithm successfully detected the wrinkle line in the nasolabial region, which compares with the performance of the proposed algorithm with state-of-the-art algorithms. Figure 8 shows that the proposed method was much better in detecting the wrinkle line compared with other shape models.

| Dataset and labeling
The FACES dataset 20

| Face feature extracting and analyzing results
Our research is based on skin patterns that is why we were faced with problems such as noise and illumination. To overcome the skin pattern noise, illumination, shadow problems Gaussian blurring, and automatic histogram equalization methods were applied ( Figure 5).

| Localize wrinkle initial position result
The wrinkle initial position was detected by calculating Eigenvalues.
The idea of the Hessian filter centers on the utilization of second-order partial derivatives for edge detection. Eigenvalues of the Hessian filter were applied to extract principal directions into which the local second-order structure of the image can be analyzed Figure 6.
Despite the fact that the Hessian filter benefits from the presence of the curve and valley extraction, a significant drawback is its omnidirectional nature. The vertical and horizontal discontinuities are detected as wrinkles; however, some of them are actually non-wrinkles. To remain only wrinkle lines, the Equation (3.10) was implemented.

| Wrinkle initial position result
In this research, for detecting the cheek wrinkle position was used for each input image unique initial shape based on edges of wrinkle line and AAM. Figure 7 illustrates two results: the first one (a) mean shape created by the appearance model and the second one (b) the mean shape created using second-order derivatives. Since for every input image applied unique initial shape based on wrinkle shape characteristics, the curve lines detected more accurately than AAM.

| Quantitative results
To compare the proposed method with state-of-the-art algorithms, ASM-based 15 where N is the total number of images 100, and J is output from Equation (4.1). If A ground truth and B output shape alignment, more than 80% decided as correct detection.

| D ISCUSS I ON AND CON CLUS I ON
The structure of wrinkles varies greatly in width, length, and pattern in different images, making it difficult to develop automatic wrinkle detection. Related works on the area of wrinkle line detection are mainly focused on detecting forehead and another part of the face based on filter and operators. Therefore, an efficient active appearance model has been proposed. This method is based on effective active appearance nasolabial wrinkle detection model, which apply the unique initial shape based on the wrinkle line structure.
In our study, was introduced the effectiveness of changing the structure of AAM and successfully applied in wrinkle line localizing.
Although competitive results are achieved by the proposed wrinkle detection method, in the future, we planned to pay attention to skin texture information that can be used to achieve to create a wrinkle mapping model. In addition to wrinkle line structure, the effects of variation of color, face alignment, and illumination shall be studied in the future works.