Variability of respiratory rate measurements in children suspected with non‐severe pneumonia in north‐east Tanzania

Abstract Objective Measurement of respiratory rate is an important clinical sign in the diagnosis of pneumonia but suffers from interobserver variation. Here, we assess the use of video recordings as a quality assurance tool that could be useful both in research and in training of staff. Methods Respiratory rates (RR) were recorded in children aged 2–59 months presenting with cough or difficulty breathing at two busy outpatient clinics in Tanzania. Measurements were repeated at 10‐min intervals in a quiet environment with simultaneous video recordings that were independently reviewed by two paediatricians. Results Eight hundred and fifty‐nine videos were sent to two paediatricians; 148 (17.2%) were considered unreadable by one or both. For the 711 (82.8%) videos that were readable by both paediatricians, there was perfect agreement for the presence of raised RR with a kappa value (κ) of 0.85 (P < 0.001); and in 476 (66.9%) cases, both paediatricians agreed on the RR within 2 breaths per minute (±2 bpm). A reported illness of 5 days or more was associated with unreadable video recordings (OR = 3.44, CI: 1.5–6.08; P < 0.001). The multilevel model showed that differences between observers accounted for only 13% of the variability in RR. Conclusion Video recordings are reliable tools for quality assurance of RR measurements in children with suspected pneumonia. Videos with a clear view of respiratory movements may also be useful in training primary healthcare staff.


Introduction
Childhood pneumonia constitutes the largest single cause of child mortality globally [1], a burden that could be substantially reduced by timely diagnosis and antibiotic treatment early in the illness. Consequently, the WHO case definition of pneumonia aims at high sensitivity whereby children with a cough or difficulty in breathing and a raised respiratory rate (for age cut-offs) are classified as having 'non-severe' or, more recently, 'fast-breathing' pneumonia, and should be treated with oral antibiotics [2,3]. However, the measurement of respiratory rate in young children is often problematic. Other clinical states such as fever or agitation may confound the association of fast breathing with pneumonia [4]. There is evidence of interobserver differences in counting respiratory rates that may account for substantial misclassification of children with suspected pneumonia [5,6]. A number of timing devices have been proposed to improve accuracy of respiratory rate measurement such as audible signals, variations in timing period and tapping a smart phone screen at each respiration [7][8][9][10]. These tools have been assessed using standard video recordings of respiratory movements, but the validation appears to be based on selection of videos where respiratory rate was clearly seen by an expert. The problems of 'real-life' measurements have been highlighted by English et al. in a review of video recordings of common paediatric clinical signs; both experts and untrained staff agreed where the sign was clearly visible and abnormal, but agreement was no better than chance where the sign was borderline or difficult to see [11]. This study did not include measurements of respiratory rate and it is not clear how common such difficulties are with respect to the measurement of respiratory rate during the process of routine care.
In spite of recommendations that measurement of the respiratory rate (RR) should be performed in a calm child, there is evidence of substantial errors in its recording that result in both overtreatment and missed diagnoses. These errors may be reduced by improved training of clinical staff in counting RR, but such training is constrained by the absence of an accessible 'gold standard' of measurement.
In a study of the influence of the clinic environment on paediatric respiratory rate, video recordings of children's respiratory movements were independently reviewed by two experienced paediatricians as a quality control exercise. We analysed these results to identify criteria that would be expected of a valid gold standard, that is that video recordings were judged to be readable and respiratory counts reached good levels of agreement.

Study area and participants
The study was conducted in the outpatient departments of two district hospitals in north-eastern Tanzania, one serving a predominantly rural population and the second serving an urban population. Details of the study participants recruitment and enrolment have been published elsewhere [12]. In summary, children aged 2-59 months presenting at outpatient clinics with cough or difficulty breathing were screened for malaria using Paracheck TM rapid test for malaria and haemoglobin levels using Hemocue TM . They were also assessed for use of anticonvulsant or sedative medication in the previous 24 h, signs of severe illness including lower chest wall in-drawing and evidence of chronic illness including asthma, and malnutrition.
The Tanzanian national guideline abides by the WHO Integrated Management of Childhood illness (IMCI) recommendations for the management of common childhood illnesses such as pneumonia [2].
Caregivers of children gave written informed consent to participate in the study. We recorded the medical history and basic socio-demographic characteristics of all enrolled children, followed by a careful physical examination including measurement of respiratory rate (RR) over one minute by research physician.
After the initial assessment at the outpatient clinic, children were transferred to a quiet room where the research nurse observed them for an hour. Every 10 min, an RR count was taken using a timer with an accompanying video recording of the child's chest movement. Each video displayed the date, time and study identification number (IDN) of the subject. Two Panasonic Lumix FS45 cameras equipped with 16 megapixels, 59 optical zoom and high definition (HD) video were used at each study site to film the child's chest (exposed). Using tripods, the camera was positioned aiming downwards on the child's chest and moved to a required angle when necessary.
The camera was switched on once the child was settled, calm or stable. If not, the child's state was documented (moving, sleeping, feeding and agitated) on the case report form. Agitation was defined as any movement and/or crying that interrupted the counting or recording of the RR. Any RR count or video recording that failed was documented and then related to the participant's characteristics. In circumstances where children could not settle at all, or the equipment malfunctioned, video recording was stopped and no video was available for later review for that child at that time point. The nurse was responsible for both RR count and camera operation and she had been trained on the WHO respiratory rate counting. Respiratory rate counting was started once the clinical state of the child was judged to be stable. The video recording of child's chest movements was read independently by two consultant paediatricians experienced in managing childhood pneumonia and teaching staff on the use of IMCI guidelines. If both paediatricians could read the video, it was classified as readable, and those not readable by either of the paediatricians were classified as unreadable. Video 'readability' was defined a priori as the paediatrician being able to view the child chest movements and count RR for 1 whole minute either easily or with some difficulty. A video counted as not readable if the paediatrician could not view and count RR for 1 whole minute.

Data management and analysis
Data were double-entered into a Microsoft Access database and statistical analyses performed using STATA version 12.0 software. Non-severe pneumonia was defined as cough or difficulty breathing with a raised RR for age, that is ≥50 breaths per minute (bpm) for those aged 2-12 months and ≥40 bpm for those aged 12-59 months.
The mean RR and 95% confidence intervals (95% CI) in RR were calculated at each time point, using the data from all children measured by the three observersresearch nurse, paediatrician 1 and paediatrician 2.
Across all time points when both paediatricians were able to read the video recording of the child's chest movement, three measures of agreement were calculated. A kappa statistic (j) was calculated for the agreement on the presence of raised RR between the two paediatricians. The agreement scores ranges from 1 for a perfect agreement to 0 for no agreement beyond chance using the classification from Landis and Koch: 0.0-0.20 signified slight agreement, 0.21-0.40 fair agreement, 0.41-0.60 moderate agreement, 0.61-0.80 substantial agreement and 0.81-1.00 almost perfect agreement [13].
The correlation coefficient was calculated on the RR measures to show the agreement between the two paediatricians in counting the RR. The third measure defined agreement as any RR where the two paediatricians were within 2 breaths per minute (AE2 bpm) of each other, which was then analysed as a binary variable using logistic regression to obtain odds ratios (OR) and 95% confidence intervals (95% CI). The AE2 bpm was chosen as representing error that could arise from including or excluding the first or last breath during the 1-min period of counting.
To assess for variability in RR count, a linear regression model was used to analyse the crude and adjusted effects across three levels of variability: children, time and observers. The coefficients (differences in RR) were calculated for the characteristics of the children, taking into account the variability between children, between time points within children and the differences between the measurements by three observers (nurse and two paediatricians) within time points.
The adjusted effect of the final model includes all the significant independent characteristics at each level. From the final model, we assess the proportion of respiratory rate variability (intraclass correlation coefficient (ICC)) at each level of the model.

Ethical approval
The study was approved by the Kilimanjaro Christian Medical Centre (KCMC) Research Ethics Committee, Tanzania. We obtained written informed consent from the parent/legal guardian of each child in the study.

Characteristics of study participants
Overall 167 children were enrolled of whom 162 (97.0%) had at least one repeat measurement of respiratory rate in quiet settings, and were used in this analysis. The mean age was 7.1 (SD AE 2.8) months in infants and 27.6 (SD AE 12.8) months in children aged 12-59 months. More males (94, 58.0%) than females (68, 42.0%) were enrolled. Other baseline characteristics of these children have been described elsewhere [12].
Comparison of respiratory rate readings between the research nurse and the two paediatricians After the initial RR count at the clinic, each study participant had RR counts at six time points in the quiet room.
For each time point, there are three RR readings available, one direct measurement by the research nurse and two by two paediatricians from the video recording ( Figure 1); 113 video records were incomplete and not available for analysis.

Readability of the videos by the two paediatricians
There were 859 available videos from 162 children, which were reviewed by the two observer paediatricians. Eighty (8.2%) videos were not readable to paediatrician 1; 142 (14.6%) were not readable to paediatrician 2 ( Table 1). In the combined results, 711 videos (82.8%) were readable by both paediatricians and 148 (17.2%) were not (Table 1). Looking at the child state, 446 (51.9%) of 859 videos were of children who were awake and calm and 210 (24.4%) videos were of children who were agitated (mild or severe), making video review difficult or impossible (Figures 2 and 3). Of the remaining videos, 152 (17.6%) were of children sleeping and 52 (6.0%) of children feeding.

Agreement on the RR counts and criteria for non-severe pneumonia
Kappa statistic for the presence of raised respiratory rate for age. In all readable videos, agreement on the presence of a raised RR (non-severe pneumonia) between the two paediatricians was 95.9% (better than the expected  Easily readable Readable with difficult Not Readable Easily readable Readable with difficult Not Readable Correlation coefficient. Overall, there was good agreement on the RR count by the two paediatricians (correlation coefficient: 0.94). However, the correlation coefficient may be influenced by RR readings at the higher and lower end of the scale, and Figure 4 shows the graphic representation of variability in RR counts.
Agreement within 2 bpm (AE2 bpm) for the RR measurement. Allowing a variation of 2 breaths per minute (bpm) as agreement in RR count between the two paediatricians for readable video recordings, there was a 33% disagreement. The expected agreement by chance is 50.2%, so 67% agreement gives a kappa of 0.34, which shows a fair agreement between the two paediatricians. Table 3 shows a proportion of the 711 video recordings that were readable within AE2 bpm and the univariate factors for agreement between the two paediatricians. In univariate analysis, agreement between the two paediatricians was lower if children were agitated (OR = 0.71, CI: 0.44-1.15; P = 0.161) and higher if they were feeding (OR = 1.88, CI: 0.77-4.55, P = 0.164). However, they were more likely to agree on RR of AE2 bpm in children with longer duration of the illness (OR = 3.44, CI: 1.95-6.08; P < 0.001). Unreadable video recordings include those videos that were available but not readable to either of the observer paediatrician.

Regression model using repeated RR measures by different observers
Random-effects linear regression was used to characterise how the observed respiratory rate varied with clinical characteristic of the children (Level 1 variables: age, body temperature, duration of illness); the effect of time (Level 2 variables: time since first measurement (in minutes), child's state) and observer (Level 3 variable). The baseline level is the RR at 10 min for children who are awake and calm, when the research nurse recorded the respiratory rate.
Univariate analysis (Table 4) showed a strong evidence of differences in RR by body temperature at baseline (Level 1), the child's state during RR measurement (Level 2) and observers (Level 3). The main difference between the observers was that the two paediatricians who reviewed the video of the child had higher estimate of the RR (coefficient: 0.53 and 0.78) than the nurse who directly measured the RR.
The multivariate analysis showed no confounding between significant variables ( Table 4). The multivariate linear regression model, with the three levels of variation (child, time and observer) explained 87% of the observed variation in the respiratory rates, leaving 13% of the variability as random error between the observers (research nurse, paediatrician 1 and paediatrician 2).
Fifty-three percentage of the variability between RR readings was due to the differences between children, and 34% due to different RR over time within the observed children.

Discussion
We aimed to assess the use of chest movement video recordings of children presenting to a busy outpatient department in order to assess the use of videos as a reference standard of paediatric respiratory rate. We used two main indicators of success; did video provide a readable result and, if so, could experienced paediatricians agree on the ensuing respiratory rate? The results demonstrated that almost one-fifth of videos did not provide a sufficiently clear view to count respirations, but when it did, agreement between expert readers was perfect (kappa = 0.85), indicating there was minimal misclassification of the WHO criteria for 'raised respiratory rate for age'. However, there was only fair agreement on the actual respiratory rate measurements within 2 breaths per minute (kappa = 0.34), perhaps indicating the limitations of using a fixed tripod for video recordings of respiratory rate measurements of children in this setting.
There is clearly a need for training and quality control tools in paediatric respiratory rate given evidence of between-observer variation in counting of respiratory rate in children with respiratory symptoms that can result in misclassification of children into the category of 'fastbreathing pneumonia' [6,14,15]. Two studies from Tanzania demonstrated only moderate levels of agreement between health workers that are likely to lead to both over-and undertreatment of childhood pneumonia [6,14]. However, these studies relied on direct observation and not video recordings. The findings have potentially important public health consequences as 'fast-breathing pneumonia' is often the commonest reason for prescribing antimicrobial drugs in paediatric outpatients [16]. Video clips of paediatric clinical signs are an important source of teaching material although there are few validation studies of such videos. Among the problems of such validation is the fact that clinical signs often lack an objective measure that can act as a quality control tool and, as in our study, 'agreement between experts' has to act as the reference standard. A study to determine the level of interobserver agreement of video recordings of 11 common paediatric clinical signs found that lack of agreement among expert readers was often the result of the sign being 'borderline' or ambiguous [15]. This is similar to the finding of our study, although respiratory rate, being counted on a continuous scale, may have less scope for interpretation but relies on a clear view of the chest movements of the child. Of relevance to clinical decision-making, the agreement on the presence or absence of raised respiratory rate was particularly good.
We analysed clinical data to identify factors associated with readability of video recordings. Longer duration of illness (≥5 days) was the most significant child characteristic associated with readable video, and agitation of the child made a readable video less likely; the difference however was not statistically significant. The latter is consistent with an earlier study to determine variability in RR measurements over time which found that RR taken when the child is agitated is the most variable and is associated with largest number of failures [17].
The high ICC obtained in our study is in accordance with the results of another recent study conducted among Mozambiquan children with the aim of determining the The multivariate analysis was performed using the variables with P values <0.05 from the univariate analysis. Final model included age (as continuous variable), temperature at baseline, history of rapid breathing, the child's state at the time of measuring the respiratory rate and readers. Intraclass variation = 87%. Bold P-values indicates significant associations reliability of RR assessment counted three times during a whole minute by independent observer paediatrician [18]. The two video observers in our study recorded higher RR than the direct measurements of the nurse, and interobserver differences accounted for 13% of the difference in RR measurements. The results show little confounding between variables, as the adjustment for the observer differences does not affect the coefficients for the other variables. Our results showed that half of the variability in the RR was due to differences between the children and a further third was due to differences over time within children.

Study limitations
Videos were all taken from the same fixed position to provide the best view of where children lay on the couch. However, an improved view may have been possible by adjusting the camera position and future studies might explore this. The fact that the nurse operated a fixed camera and also did the counting of respirations may have confounded the RR counts of the nurse even though this was not a common occurrence as any disturbance led to repeat or discontinuation of the recording and RR count. Similarly, videos were taken at predefined time intervals as the primary aim of the study was to document how respiratory rate varies after placing children in a quiet environment designed to produce maximum settling effect. We recognise that variation in body temperature may influence respiratory rate and this could have influenced the readability of video records [4]. Another limitation was that although we have used consultant paediatricians as a reference standard, they were not retrained in IMCI (WHO) respiratory rate count standards.

Conclusion
Video recordings of respiratory rate in African children are constrained by the difficulty in obtaining a clear view of respiratory movements in up to one-fifth of children with suspected pneumonia. More flexible use of the camera position may improve on this. Where a clear view is obtained, video is a valid tool for staff training and quality control purposes in clinical research. Future studies are needed to refine the methods used in our study.