Objective and automatic grading system of facial signs from selfie pictures of South African women: Characterization of changes with age and sun‐exposures

Abstract Objective To evaluate the capacity of the automatic detection system to accurately grade, from smartphones’ selfie pictures, the severity of fifteen facial signs in South African women and their changes related to age and sun‐exposure habits. Methods A two‐steps approach was conducted based on self‐taken selfie images. At first, to assess on 306 South African women (20–69 years) enrolled in Pretoria area (25.74°S, 28.22°E), age changes on fifteen facial signs measured by an artificial intelligence (AI)‐based automatic grading system previously validated by experts/dermatologists. Second, as these South African panelists were recruited according to their usual behavior toward sun‐exposure, that is, nonsun‐phobic (NSP, N = 151) and sun‐phobic (SP, N = 155) and through their regular and early use of a photo‐protective product, to characterize the facial photo‐damages. Results (1) The automatic scores showed significant changes with age, by decade, of sagging and wrinkles/texture (p < 0.05) after 20 and 30 years, respectively. Pigmentation cluster scores presented no significant changes with age whereas cheek skin pores enlarged at a low extent with two plateaus at thirties and fifties. (2) After 60 years, a significantly increased severity of wrinkles/texture and sagging was observed in NSP versus SP women (p < 0.05). A trend of an increased pigmentation of the eye contour (p = 0.06) was observed after 50 years. Conclusion This work illustrates specific impacts of aging and sun‐exposures on facial signs of South African women, when compared to previous experiments conducted in Europe or East Asia. Results significantly confirm the importance of sun‐avoidance coupled with photo‐protective measures to avoid long‐term skin damages. In inclusive epidemiological studies that aim at investigating large human panels in very different contexts, the AI‐based system offers a fast, affordable and confidential approach in the detection and quantification of facial signs and their dependency with ages, environments, and lifestyles.


INTRODUCTION
The efficacy of cosmetic products can be rationally assessed in vivo under three complementary approaches: instrumentally (when possible), clinically and responses from the human panel under test. A good coherence between these three domains thus confers to the product efficacy a strong relevance. This paradigm seems however entering into a new era that concerns both consumers and/or clinical studies as the consumer under test becomes prone at supplying and checking objective data. Such is the recent case of automatic grading system for detection and quantification of facial signs from selfie pictures, thanks to smartphones with powerful hardware and high-resolution cameras.
As for now, mostly focusing on facial images, these allow to rapidly obtain digital data on large cohorts, in specific environments or lifestyles. Hence, these new embarked automatic grading systems that detect and quantify of facial signs are powerful methodologies to better depict and quantify the slow process of facial skin aging, under reallife conditions, on large panels of humans of various ages, backgrounds and habits.
These systems basically ground on artificial intelligence (AI) and deep-learning-based algorithms dedicated to medical purposes 1-3 or cosmetical applications. As examples, an AI-based automatic grading system 4,5 allowed to attribute the grading of the severity of facial signs using skin aging atlases 6,7 taken as a reference, on differently aged women with dark skin tones. This algorithm has been trained and validated versus scoring obtained from experts and dermatologists.
This system was applied to changes in skin color, related to age, geographical locations, and environmental conditions, that is, the so-called exposome. 8 Of note, skin aging leads to cultural impacts as some facial signs own different weights in the overall perception of a younger or older aspect, as demonstrated in different locations such as South Africa. 9,10 However, with regard to constitutive dark skin tones, the relative impacts of photo-aging 11 and photo-protection, 12 TA B L E 1 Questionnaire used to evaluate the history of sun-exposures (combined or not to a photo-protection habit) of the South African women enrolled in the study. In bold, the numbers used for calculating the sun history and habits index (SHHI)

Have you ever spent more than 4 h a day outdoors for professional or recreational activities
Between 0 and 7 years old. . .

Subjects and conditions of privacy in the collection of smartphones' selfie pictures
Three hundred six South African women of different ages (20−69 years) with photo-types V and VI were recruited in Pretoria (25.74 • S, 28.22 • E), where all permanently reside since more than 15 years. Based on the clustering method described above-already published 18-21 -two groups were identified according to their own sunexposures habits, that is, sun-phobic (SP) and nonsun-phobic (NSP).

2.2
Clustering procedure for sun-phobic and nonsun-phobic groups To establish their usual relationships with sun-exposures, subjects were asked to fill a dedicated questionnaire ( Table 1). The latter aimed at establishing a sun history and habits index (SHHI) by taking into account their daily sun-exposures (>4 h a day) and the regular use (or not) of a photo-protective cosmetic product. An index is obtained by multiplying, in each age-range, the numbers (1, 2, or 3) of the exposure by those (1 or 2) corresponding to the use of a photo-protective product. The global SHHI is obtained by adding all four indexes, further divided by 4. Hence, this index varies from 1 to 6, where 1 corresponds to the least degree of sun-exposures ([1 + 1 + 1 + 1]/4) and 6 ([6 + 6 + 6 + 6]/4) as the most extreme case being a regular sun-exposure at young ages, devoid of photo-protection. In such calculated index, value of 3.5 represents women who are regularly exposed to sun with a systematic photo-protective habit, that is, the intermediate case.
TA B L E 2 Questionnaire used to evaluate the equivalence of two cohorts created with sun history and habits index (SHHI) on other internal and external exposome factors for the Japanese women enrolled in the South African study Hence, the threshold between SP and NSP subjects was arbitrarily set at 3.5. These clustering criteria led to the creation of two groups, that is, sun-phobic (N = 155) and nonsun-phobic subjects (N = 151) women, well balanced in ages and SHHI-see above. Table 2 gathers other questions related to "exposome" 8 to establish equivalence among the two groups on all other factors, behavior toward sun excepted.

Protocol
The protocol was tiered into three successive steps: Step 1: All 306 subjects were asked to take one selfie image (full face, frontal camera of smartphone) at morning, under the conditions exposed above.
Step 2: Selfie images were analyzed by the automatic grading system algorithm, 4,5 and the resulting scores of the 15 facial signs (Table 3) were sent to our secured website under blind codes.
Step 3: Automatic scores were analyzed to characterize agerelated changes or sun-exposures habits impacts.

Facial signs assessed by dermatologists and by automatic grading system
The interest of using standard photographic scales to bring reliability and robustness in clinical evaluation of darkest skin tones was previously discussed. [22][23][24] Table 3 illustrates the 15 facial signs (definitions and respective scoring ranges) that were further analyzed by the AIbased automatic grading system. The latter, based on smartphones' selfie images on South African subjects, uses the standardized grades of the skin aging atlases as reference. 6 The training of the AI-based automatic grading system dedicated to women with darkest skin tones and its validation versus experts and dermatologists has been previously described. 4,5 In short, it is a super-F I G U R E 1 Two illustrative smartphone's images of two South African women enrolled in the study vised regression problem within deep learning framework based on convolution neural networks.

2.5
General characteristics of the cohorts and conditions of privacy in the collection of smartphones' selfie pictures Table 4 summarizes the distribution of subjects by age-classes and sunexposures habits.
The two groups were at best balanced in ages-classes following some imperative inclusion criteria (Table 4). These were as follows: (1) possessing a smartphone (any brand) with a high-resolution camera (≥5 Megapixels), (2) used to take selfie pictures, and (3) free from any facial skin disease, disorder or scars (rosacea, acne, angioma, etc). All subjects were fully informed about the objective of the study and signed an informed consent. The latter guaranteed that their photographs were totally confidential (blind-coded) and further deleted once Cheek pores (one sign) Cheek skin pores Size of visible pores on the cheek irrespective of their densities.  Table 5 illustrates the progressive changes of the 15 facial signs, and Figure 2 shows the four clinical-associated clusters (mean grades ± confidence interval) with age. Globally, the changes on lifespan remain low in term of grading units compared to other populations. 25 Wrinkles/Texture and sagging appear of a rather regular rate (p < 0.05) after 20 and 30 years respectively whereas pigmentation shows no changes with age. Surprisingly, individual pigmentation signs evaluated by AI-based automatic grading system show erratic changes among nature of disorders, that is, no changes for pigmentary exgrowth or in depigmented surface, decreased density of dark spots TA B L E 5 Changes of 15 facial signs with ages in 306 South African women (Mean ± confidence interval 95% [CI]) measured by artificial intelligence (AI)-based automatic grading system. Green or grey colors cells correspond to nonsignificant differences between them. Each white cell is significantly different from all others  Table 6 gathers the significant and relevant differences among SP and NSP groups for the fifteen facial signs assessed by the AI-based automatic grading system according to age-classes. The impacts of sun-exposures on South African panelists were significantly found after 60 years old. These lead to significant and large increase in wrinkles/texture signs (nasolabial fold, marionette lines, cheek folds, texture of mouth contour) and sagging (ptosis of the lower part of the face). Interestingly, in the fifties, despite rather low amplitude of changes in pigmentated signs, close to being significant, an impact of sun-exposures in the pigmentation of the eye contour was detected.

Answers from questionnaire
Attempts to differentiate the SP and NSP groups through their conditions and duration of transportation and/or their alcohol and smoking habits ( Table 2) failed. Such negative findings suggest that the significant differences, obtained by AI-based automatic grading system analysis are, in a large part, most linked to the different sun-exposure habits.

DISCUSSION
The methodology using selfie images coupled to AI-based automatic grading is a promising approach, previously validated versus experts and dermatologists gradings on Asian, African, and Caucasian skins. 4,5 The latter study 5 indeed showed significant agreements between experts and automatic gradings of all facial signs (wrinkles/skin texture, sagging, cheek pores etc.) although less significant in pigmentation signs of dark skin complexions where the contrast between a small TA B L E 6 Changes in the 15 facial signs scoring observed between sun-phobic (N = 155) and nonsun-phobic (N = 151) behaviors for each age-class. Differences between these values represent the impact of sun exposures upon clinical signs severity. In regular black, those where differences did not statistically differ. In bold green, all signs that present a significant (p < 0.05) negative difference for nonsun-phobic (NSP), as compared to SP. In bold red, all signs that present a significant (p < 0.05) positive difference ≥0.20 grade for NSP, as compared to SP. In regular red, all signs that present a limit significant (p ≤ 0.10) positive difference ≥0.15 grade for NSP, as compared to SP. L, limit significant; NS, not significant; S, significant The present study globally found a lower dynamic of facial skin aging in African skin than that of other ethnicities, 25  It was observed here that the changes in the signs of pigmentation changes with ages, and sun-exposures were rather erratic. Such finding highlights the need to being more exhaustive, that is, to follow more specific markers in darkest skin tones for the reasons exposed above. Despite some limitations in the precision of changes in skin pig-mentation, the present study nevertheless indicates that these are of very small amplitudes (close to zero) along the life span ( Figure 2), inversely to other signs (wrinkles, sagging, cheek pores) that continuously increase by rather small steps between decades. In short, the facial skin aging of our studied cohort appears discrete and much more driven by skin surface events than its pigmentary aspect.

Age-clusters/Facial
According to literature, [29][30][31][32][33] if visible manifestations of aging or photo-aging of dark-skinned people are delayed versus other populations and especially in wrinkles/texture and sagging, the case of changes in the skin structure and its mechanical properties remains unknown. To such objective, coupling digital studies to video recordings of a standardized movement could probably be able to assess the structural alterations in dynamic behavior of facial skin occurring with aging. 34 We acknowledge however that this study presents certain lim-