Handheld fundus camera performance, image quality and outcomes of diabetic retinopathy grading in a pilot screening study

To compare the performance and image quality of the handheld fundus camera to standard table‐top fundus cameras in diabetic retinopathy (DR) screening. The reliability and diagnostic accuracy of DR grading performed by an ophthalmologist and a photographer reader were evaluated.


Introduction
Diabetic retinopathy (DR) is still one of the primary causes of vision loss worldwide (Leasher et al. 2016). Almost half a billion people are diagnosed with diabetes globally, and the number is expected to increase to 700 million people by the year 2045. Moreover, half of the people living with diabetes lack diagnosis (Saeedi et al. 2019). At least, some form of DR is estimated in 35% of the patients with diabetes and 10% of them have visionthreatening DR (Yau et al. 2012). DR is, however, often symptomless even in severe stages when treatment would be necessary in order to avoid further development of sight-threatening complications. Fortunately, even early changes of DR can be detected by fundus examination, and timely intervention may prevent or reduce vision loss. It is, thus, important that patients with diabetes are screened regularly in case of DR by ophthalmic examination or, more frequently, by fundus photography.
In Finland, each patient with either type 1 or type 2 diabetes is entitled to regular screening by fundus photography according to the Current Care Guideline for DR (Summanen et al. 2015). The screening system in Northern Finland has been further developed by organizing a mobile eye examination unit named EyeMo notably for the rural areas in order to provide equal access to screening for all patients (Hautala et al. 2009). The earlier detection of DR by more effective screening and early treatment has led to the substantial, almost 90%, reduction of visual impairment in patients with diabetes in Northern Finland (Hautala et al. 2014). Thus, DR screening is not only cost-effective, but it may even save costs due to significant expenses caused by visual loss.
Retinal photography by standard table-top fundus cameras and manual reading of images by an ophthalmologist or a trained health-care professional are common approaches in screening of DR (Lanzetta et al. 2020). A handheld and easily movable fundus camera might serve as an alternative tool for organizing screening of DR in a costeffective manner (Davila et al. 2017). It is possible to speed up the screening process by grading the stage of DR by a photographer immediately after shooting the images. Thus, we compared the performance of the portable Optomed Aurora fundus camera to that of traditional table-top fundus cameras for image quality and DR detection in a pilot study. The sensitivity, specificity and reliability of DR grading by an ophthalmologist and an experienced photographer were also evaluated.

Materials and Methods
This study was carried out at Oulu University Hospital and its mobile eye research unit EyeMo. The study followed the tenets of the Declaration of Helsinki, and it was conducted with the approval of the Oulu University Hospital Research Committee.
A total of 157 patients were included in the pilot study. In the first phase, 107 consecutive patients with either type 1 or type 2 diabetes attending screening of DR in EyeMo were evaluated. The black-and-white 45°fundus photographs were taken by an ophthalmic photographer first with the traditional table-top fundus camera (Canon CF-1) after pupil dilation with 1% tropicamide. Mydriatic was used to improve the image quality. Consecutively, another set of colour and black-andwhite fundus images were taken with the handheld Optomed Aurora fundus camera in mydriasis (Optomed Aurora, Optomed, Finland). The impact of patient collaboration during the photography was considered equal, since fundus images by separate cameras were taken one after another in identical conditions. This portable camera has a 50°field of view, non-mydriatic operation, nine internal fixation targets and Wifi for transmitting images to the PC. Secondly, another 50 patients with more severe cases of DR and other retinal changes (age-related macular degeneration, retinal vein occlusion, etc.) attending follow-up visits in the hospital's outpatient eye clinic were examined. 45°fundus photography was performed after mydriasis with a tabletop fundus camera (Zeiss Visucam 524), and 50°photography was performed with Optomed Aurora by the same photographer. From each patient, a total of twelve fundus images taken by separate cameras were included in the study (black-and-white papilla-and macula-centred images by the table-top and handheld cameras and additional colour papilla-and macula-centred images by the handheld camera from both eyes of each patient). Each set of two-field images were analysed by using the high-quality 27" screens. Demographics were not collected (Fig. 1).
The quality of the black-and-white images taken by handheld or table-top cameras was graded according to the criteria described earlier by Boucher et al. (Boucher et al. 2005). Quality grades 1, 2, and 3 were considered as sufficient for interpretation ( Table 1). Classification of the severity stage of DR was performed using the five-scale grading system developed by the Finnish Current Care Guidelines as shown in Table 2 (Summanen et al. 2015). The stages 2 (moderate background retinopathy), 3 (severe background retinopathy) and 4 (proliferative diabetic retinopathy) were considered as referable in the DR screening programme. The overall level of DR of each patient and the need for a referral to ophthalmologist were determined from the stage of DR in an eye with more severe changes. The ophthalmologist's grading of DR in black-andwhite images taken with the traditional table-top fundus camera was used as a gold standard when calculating the sensitivity, specificity and reliability of the reading outcomes. The severity of DR and the image quality were analysed independently by the ophthalmologist reader and the photographer. In addition, any other retinal abnormalities were also documented for attention. The images from each camera were graded independently at different time points, and all patient information was anonymous for the grader.

Statistical analyses
A paired-samples t-test was used to calculate the mean difference in quality assessment from the same image between the photographer and the ophthalmologist. Sensitivity and specificity were calculated to evaluate diagnostic accuracy. Furthermore, we calculated the weighted kappa (j) to evaluate the agreement of the degree of DR. The kappa coefficient was calculated according to Yang & Zhou (2015), where the clustering of the data (within-patient-eyes are more related than between-patient-eyes) was taken into consideration. The interpretation in regard to the kappa coefficient is as follows: j < 0 as poor, 0.00-0.20 as slight, 0.21-0.40 as fair, 0.41-0.60 as moderate, 0.61-0.80 as substantial and 0.81-1.00 as almost perfect. A ninetyfive % confidence interval (95% CI) is presented for mean difference, j, sensitivity and specificity. Weighted j was calculated by means of SAS macro provided by Yang

Results
A total of 1884 fundus images taken with the commonly used table-top fundus cameras (Canon CF-1 or Zeiss Visucam 524) and Optomed Aurora were analysed. 107 out of the 157 patients included in the study were attending DR screening, and the remaining 50 patients were attending follow-up visits at the hospital's outpatient clinic as a result of more advanced DR. 68% of the eyes of the patients attending regular DR screening had no DR, 13% had mild non-proliferative diabetic retinopathy (NPDR), 13% moderate NPDR and 1% severe NPDR. Only, 4% of the eyes of the screening patients had proliferative diabetic retinopathy (PDR). Out of all the eyes in the study, more than a half (53%) had no signs of DR. Mild NPDR was detected in 10%, moderate NPDR in 16%, severe NPDR in 6% and PDR in 16% (Table 3). Detected retinal abnormalities other than DR were choroidal nevus (17 patients), age-related macular degeneration (10 patients), central retinal vein occlusion (4 patients), branch retinal vein occlusion (2 patients) and macular pucker (2 patients).
The DR grading outcomes of the images taken by Optomed Aurora equalled highly to those of Canon or Zeiss (Table 4). There is almost perfect agreement in identifying DR from Aurora fundus images by the ophthalmologist (j = 0.93, 95% CI 0.91 to 0.96) and the photographer (j = 0.89, 95% Cl 0.85 to 0.93) within the reference standard. In addition, there was almost perfect agreement (j = 0.95, 95% Cl 0.93 to 0.98) with the ophthalmologist and the photographer in identifying DR from fundus images taken by the table-top camera.
The ophthalmologist and the photographer had sensitivities of 91.8% (95% Cl 97.8 to 100) and 91.2% (95% Cl 85.4 to 95.2) when detecting any level of DR from Optomed Aurora fundus images. Specificities were 100% (95% CI 97.8 to 100) and 100% (95% Table 1. The fundus image quality analysis was performed using a 5-grade classification, and the grades 1, 2, and 3 were considered to be of sufficient quality for interpretation for diabetic retinopathy. The criteria for each quality level have been described earlier (Boucher et al. 2005).

Grade
The criteria for the classification of the image quality Quality for interpretation 1 Excellent definition Sufficient for interpretation (grades 1-3) 2 Good definition of most details 3 Definition limited but permitting confident assessment 4 Insufficient definition for confident assessment Not sufficient for interpretation (grades 4-5) 5 Only gross or no details visible   Table 5). The quality of the black-and-white fundus images taken by the table-top fundus cameras and Optomed Aurora was 1.4 and 2.5 when graded by the ophthalmologist, respectively. The photographer judged the quality of images taken with the Optomed Aurora fundus camera to grade 2.3. A majority, 84% and 88%, of the images taken by Optomed Aurora fulfilled the criteria of sufficient quality for interpretation analysed by the ophthalmologist and the photographer, respectively. The quality of both macula-and papilla-centred fundus images was similar. Three per cent of these patients could not be reliably photographed by either commonly used table-top fundus cameras due to insufficient fundus visibility (cataract, corneal or vitreous haze etc.).

Discussion
The results of the current pilot study show that the diagnostic performance  of a handheld Optomed Aurora fundus camera might be appropriate for the screening of DR. Moreover, the reliability of DR grading performed by the photographer was excellent, suggesting an opportunity to improve the DR screening process with immediate reading of DR after shooting the images. Screening for DR is an important but a substantial cost-effective aspect of diabetes management (Rohan et al. 1989;Javitt & Aiello 1996). Previous clinical trials have shown that up to 98% of visual loss due to DR could be prevented by timely treatment (Cheung et al. 2010;Tan et al. 2017). Improvements of screening programmes for early detection and intervention for DR are particularly important during the present era due to the enormous increase in the prevalence of diabetes as well as DR worldwide (Yau et al. 2012;Saeedi et al. 2019).
Retinal photography by standard table-top fundus cameras and manual reading of images by an ophthalmologist or a trained nurse are common approaches in screening of DR (Hautala et al. 2009, Lanzetta et al. 2020. Novel imaging technologies, such as wide-and ultrawide-field fundus cameras, would broaden the possibilities of fundus image screening. However, the size, weight and the cost of the traditional or novel table-top cameras set some limitations to their large-scale use, especially in areas lacking a sufficient operative screening process. A handheld and easily movable fundus camera might serve as an alternative and cost-effective tool for organizing screening of DR especially in countries with low health care and resource levels (Abegunde et al. 2007;Panwar et al. 2016;International Diabetes Foundation 2017). The Optomed Aurora fundus camera seems to perform appropriately in screening of DR with a sensitivity of 91.8% (95% CI 86.2% to 95.7%) and a specificity of 100% (95% CI 97.8 to 100%) in the current study reaching the targets of 80% in sensitivity and 90% to 95% in specificity for DR assessment which has been widely recommended by international retina experts (British Diabetic Association 1997; Canadian Ophthalmological Society Diabetic Retinopathy Clinical Practice Guideline Expert Committee et al. 2012).
The quality of Optomed Aurora black-and-white images fulfilled the criteria of sufficient quality for interpretation in a majority, 84-88%, of the cases in the present study. This is in line with the previous study, where 12% of the fundus images taken by a non-mydriatic camera (iCam-Optovue) after pupil dilation showed insufficient image quality to allow reading (Boucher et al. 2020). It is notable, though, that 97% of the black-and-white images taken with the traditional table-top fundus cameras in the present study reached the level of the sufficient quality for reliable grading of DR. The quality of the images was reduced in 3% of the patients with cataract and corneal or vitreous haze despite the type of the fundus camera used. A previous study suggested that the level of training of the fundus photographers might not affect the quality of images (Maberley et al. 2004). In our study, a notable amount (97%) of images taken by an expert photographer were of good quality, suggesting that the experience of the photographer might be crucial for the high frequency of the availability of qualified fundus photographs. The expertise of a photograph reader is also emphasized in the screening process according to the reliable detection of retinal findings other than DR. In our study, no cases of misreading by the photographer occurred in patients requiring referral to an ophthalmologist due to retinal occlusions or wet agerelated macular degeneration.
Despite numerous recommendations and broad consensus of the importance of DR screening, adherence to regular screening is inadequate in many communities, possibly due to lack of resources, an effective screening programme or trained health-care professionals. Manual screening using fundus photographs involves considerable costs for both the patients and the entire health-care system, which may have limited its application at least in lowincome settings. Telemedicine-based screening programmes with digital fundus photography have managed to increase the demand for DR screening, in addition to telemedicine technology having the potential to be cost-effective (Hautala et al. 2014;Kalogeropoulos et al. 2020;Avidor et al. 2020). However, artificial intelligence (AI) and several deep learning technologies have recently raised the possibility of a widespread automated screening system in the future (Lim et al. 2020;Vujosevic et al. 2020;Chaudhary 2020;Shah et al. 2020). Still, more research is needed to address the potential clinical deployment and costeffectiveness of such solutions before being widely accepted in the everyday clinical practice.
Meanwhile, any tools which improve DR screening and management will help in decreasing the burden of tackling this potential cause of blindness. In this study, we evaluated whether an ophthalmic photographer could read fundus images and assess DR grading immediately after shooting the photographs. An almost perfect agreement (j = 0.96 (95% CI 0.95 to 0.97)) was noted in identifying the presence and severity of DR by the ophthalmologist and the photographer. Screening sensitivities of 100% and 100%, and specificities of 98.7% and 98.5% were obtained by the photographer in detecting any DR or identification of referable DR, respectively. Boucher et al. have revealed comparable 91% and 97% sensitivities, and 95% and 85% specificities in detecting any DR, and 86.8% and 91.2% sensitivities, and 91.7% and 97.0% specificities in detecting sight-threatening diseases by two nurse DR readers (Boucher et al. 2020). A similar DR grader sensitivity of 92.5% and specificity of 88% was reported by Goatman et al. (2012). Interestingly, recent studies by Bhargava et al. and McKenna et al. showed better diagnostic accuracy in DR grading among non-medical graders compared to that of family physicians or rural doctors (Bhargava et al. 2012;McKenna et al. 2018). These results are in line with our suggestion to improve DR screening by involving a non-medical photographer reader in the process as the first-line grader. The resources for screening would thus be used efficiently, and the screening process would speed up if the stage of DR would be graded immediately after shooting the images.
The limitations of this pilot screening study include the small number of graders, which only validates the individual performance of these readers, and the results may not apply to other readers due to the possible variations in performance and experience. However, several countries have clear recommendations for classification of DR and guidelines for referral to ophthalmologist that helps to standardize the interpretation in DR screening (Summanen et al. 2015;Wong et al. 2018). To further validate the use of the handheld camera in regular DR screening, several screening centres and graders should be involved. The strength of the study is a relatively high amount of fundus images included in the comparative evaluation of the two imaging methods. Study patients include those attending regular screening for DR, but also those with more severe DR and other retinal abnormalities followed-up in hospital. The performance of Optomed Aurora would thus have been even better for just sorting out healthy eyes from those with any DR in the pure screening setting.
In conclusion, our results show that the handheld Optomed Aurora fundus camera performs well in DR screening according to the high sensitivity and specificity of DR detection as well as the image quality. We suggest that photographers could be more involved in DR grading leading to fewer delays in the screening process. This would not only improve the screening process, but also save time and reduce the need for resources, in addition to lowering expenses of DR screening with no compromises in the reliability of the outcomes.