Electrolyte measurements differ between point-of-care and reference analysers in dogs with hypoadrenocorticism

I ntroductIon : Dogs treated for hypoadrenocorticism are monitored through analysis of their blood electrolytes. This is routinely performed with point-of-care analysers and doses of medications are adjusted based on the results. o bjectIves : To investigate the performance of two point-of-care analysers (IDEXX Catalyst Dx and IDEXX VetStat) against a reference laboratory method for the measurement of blood sodium, potassium and chloride concentrations, as well as sodium: potassium ratios, in dogs diagnosed with and treated for hypoadrenocorticism. M ethods : Forty-eight dogs were enrolled into a prospective cross-sectional study. Paired blood samples were taken and tested on two point-of-care analysers and at a reference laboratory. Statistical analysis was then performed with Bland-Altman analysis and Passing-Bablok regression. The clinical effects of inaccurate electrolyte analysis were investigated. r esults : In total, 329 samples were tested on the Catalyst analyser, while another 72 samples were tested on the VetStat. Passing-Bablok regression identified both proportional and constant bias for some analytes. There was poor agreement between sodium and chloride concentrations on both analysers. Both analysers tended to give higher results than the reference method for all analytes, except for potassium when measured on the VetStat. c lInIcal s IgnIfIcance : There are inherent differences between the electrolyte concentrations measured by these two point-of-care analysers and reference laboratory methods in dogs with hypoadrenocorticism.


INTRODUCTION
The accurate measurement of electrolytes is important in the diagnosis and management of many conditions in veterinary medicine. Electrolytes may be most accurately measured at a reference laboratory; however, this is not always practical for patient management. More often electrolytes are measured in the clinic using point-of-care (POC) analysers to allow for faster diagnoses and adjustments to treatment plans. However, inaccurate POC electrolyte measurement may lead to delayed or inappropriate therapy which can have clinically detrimental effects on patients. Dogs receiving treatment for hypoadrenocorticism are typically ttp://www.bsava.com/ monitored by analysis of their plasma, serum or whole blood sodium concentration [Na + ], potassium concentration [K + ] and sodium: potassium (Na + : K + ) ratio in conjunction with their clinical signs. Chloride concentration [Cl − ] is usually assayed along with these other electrolytes on POC analysers.
There are several methodologies available for electrolyte analysis (Schindler et al. 2018). Flame photometry (FP) is the recognised reference method; however, it is rarely used outside of a research setting due to inherent impracticalities in performing the test. Indirect potentiometry by an ion-selective electrode (ISE) is therefore used in most commercial reference laboratories instead and produces results comparable to FP. Several POC electrolyte analysers are available for veterinary use and commonly employ either direct potentiometry by an ISE, enzymatic spectrophotometry (ES) or optical fluorescence (OF). All methods of electrolyte analysis are vulnerable to interference from various physiological abnormalities. The indirect ISE method involves sample dilution before analysis and as such is particularly sensitive to endogenous interferents such as high levels of lipid or protein which may falsely decrease electrolyte concentrations. This affect is most apparent with [Na + ] but may occur with all electrolytes. Similarly, hyperbilirubinaemia and haemolysis will also interfere with electrolyte analysis (Stockham & Scott 2008, Bernardini et al. 2009, Scott et al. 2015, Schindler et al. 2018. The Catalyst (IDEXX Catalyst Dx, IDEXX Laboratories, Westbrook, USA) is one of the most widely used POC analysers and employs an ES methodology incorporated into "dry slide technology." These slides have several layers of filters which attempt to minimise interference caused by physiologic abnormalities (Hubl et al. 1994, Bernardini et al. 2009), although ES remains more susceptible to the effects of interferents than the indirect ISE method. The VetStat (IDEXX VetStat, IDEXX Laboratories, Westbrook, USA) is another common POC analyser but it uses OF measured via sensors called optodes. The test is incorporated into a disposable single-use test cassette along with information for calibration.
A recent survey of veterinary practices found that almost half (45%) of the respondents' POC electrolyte analysers had no quality assurance testing performed (Bell et al. 2014). Several previous studies have indicated that POC analysers show error and bias with many different analytes (Rishniw et al. 2012, Baral et al. 2015b. Of the electrolytes measured, [Na + ] and [K + ] were more prone to error. This error may be further compounded when Na + : K + ratio monitoring is used as this combines the error of both analytes. Previous studies have also shown POC analysers to have poor agreement and correlation when measuring electrolytes (Papasouliotis et al. 2008, West et al. 2014, Uyanik et al. 2015. However, these studies have variably employed different methodologies in the POC analysers investigated i.e. direct ISE methodology, or have used either whole blood, serum or in some cases, quality control material (Baral et al. 2016). Most method comparison studies have also been focused on clinical biochemistry analysis rather than electrolytes (Trumel et al. 2005, Irvine et al. 2016. Other studies have focused on non-canine species, including cats (Baral et al. 2015a, c), cattle (Yildirim et al. 2015), horses (Kirsch et al. 2019) and rabbits (Selleri & Di Girolamo 2014, West et al. 2014. POC analysers are frequently used for electrolyte analysis in dogs with hypoadrenocorticism as their faster analysis times allow dose adjustments to be made during the consultation compared to waiting for results from a reference laboratory. It was our clinical impression that the [Na + ], [K + ] and Na + : K + ratio measured from our POC analysers were often quite different from the reference laboratory results. A small variation was expected given the difference in analyser reference ranges (Table 1) and methodologies, but occasionally the difference between the results of the different methods were sufficiently dissimilar that different clinical decisions may have been made.
The purpose of this study was to investigate the performance of the Catalyst and VetStat analysers against a reference laboratory indirect ISE method for the measurement of [Na + ], [K + ], [Cl − ] and Na + : K + ratios in the plasma and whole blood of dogs with hypoadrenocorticism. Our null hypotheses were that the Catalyst and VetStat would have good precision over a wide range for sodium, potassium and chloride concentrations, that they would have high correlations and good agreement with results from the reference ISE method and that any identified bias between the Catalyst or VetStat and ISE would not impact on clinical decisions.

Ethical approval
Institutional ethical approval was granted by the School of Veterinary Medicine Ethics and Welfare Committee, University of Glasgow (REF03a/17) using blood samples collected for routine clinical purposes. Some of the dogs were also enrolled into a clinical trial under an Animal Test Certificate (10434/0002) with informed owner consent.

Subjects
Forty-eight dogs diagnosed with hypoadrenocorticism and treated at the Small Animal Hospital were enrolled into a prospective cross-sectional study between October 2015 and April 2019. Samples were included from dogs during all stages of treatment, Sampling and laboratory methods Samples were collected from each dog by jugular venepuncture into 1.3 mL lithium-heparin tubes. Samples run on the Catalyst analyser were centrifuged at 9000 rpm for 3 minutes and 300 μL of plasma was analysed within 10 minutes of collection. The Vet-Stat analyser used 200 μL of heparinised whole blood drawn into a 1 mL syringe. Both analysers were installed to the manufacturer's specifications and serviced as recommended with daily and monthly quality control checks using the manufacturer's quality control products (IDEXX Laboratories 2010, 2019). Results were shared anonymously with IDEXX's SmartService allowing for automatic software updates, remote calibration and analysis.
In compliance with the manufacturer's instructions, Catalyst test clips (Lyte 4 CLIP, IDEXX Laboratories, Westbrook, USA) were kept at −20°C until immediately prior to testing while the VetStat cassettes (Electrolyte 8 Plus, IDEXX Laboratories, Westbrook, USA) were kept between 4 and 30°C, at room temperature. Samples sent to the reference laboratory for testing were analysed within 24 hours of sampling representing standard clinical practice. The reference laboratory (Veterinary Diagnostic Services, University of Glasgow) utilised an indirect ISE methodology using 40 μL of plasma diluted 1:10 and tested on a Siemens Dimension Xpand Plus (Siemens Healthcare Ltd, Surrey, UK). This analyser was installed and maintained to the manufacturer's specifications and recommendations. The ISE component underwent calibration every 4 hours with internal quality controls performed twice a day, external quality controls performed monthly and routine servicing every 4 months as recommended by the manufacturer.

Statistical analysis
Statistical analysis was performed using commercially available statistical software (Analyse-It Software Ltd, Version 5.40, Leeds, UK and GraphPad Prism 8, San Diego, USA). The data were assessed for normality by the Shapiro-Wilk test and by visual inspection of graphical plots. As none of the data was normally distributed, non-parametric statistical tests were used for all comparisons. The correlation between methods was calculated using the Spearman's rank correlation coefficient (r).
The lack of agreement was assessed by calculating the bias and displayed using Bland-Altman plots for each variable. The differences were not normally distributed so 95% limits of agreement (LOA) were derived based on the 2.5th and 97.5th percentiles around the median (Altman & Bland 1983, Bland & Altman 1986, 1999, Ludbrook 2010. Agreement was considered good when the LOA were within the American Society of Veterinary Clinical Pathology (ASVCP) recommended allowable total error (TE A ). For [Na + ] and [Cl − ], this is ±5% of the target value and for [K + ] it is ±5% or ± 10% at low levels. Alternatively, the clinical laboratory improvement amendment (CLIA) guidelines suggest a TE A of ±4.0 mmol/L for [Na + ] and ± 0.5 mmol/L for [K + ] (Harr and others 2013) so both values were considered.
Cohen's kappa coefficient (κ) statistics were calculated to further assess agreement between methods. Results were binned into three categories ("low," "normal" and "high") based on if they fell below, within or above the reference range for each analyser. These were then compared to the ISE analyser. It was considered that the agreement was poor if κ < 0.2, fair if κ = 0.21 to 0.40, moderate if κ = 0.41 to 0.60, good if κ = 0.61 to 0.80 and very good if κ > 0.81 (Landis & Koch 1977).
The inter-assay imprecision of both the Catalyst and VetStat analysers was assessed by repeated analysis of three samples run five times consecutively on each analyser. Different levels of [Na + ], [K + ] and [Cl − ] were studied. This allowed determination of the coefficient of variation (CV), bias and intraassay observed total error (TE Obs) ) for each analyte on both analysers. The TE (Obs) was defined as Bias (%) + 2CV. The TE (Obs) was then compared to the TE A of each analyte to determine if it was acceptable according to the ASVCP guidelines (Flatland et al. 2014).
All analytes showed a linear correlation so Passing-Bablok linear regression analysis was performed for method comparison to investigate bias. With Passing-Bablok regression, if the Y intercept differed significantly from 0 it was considered constant bias was present. Similarly, proportional bias was considered to exist if the slope did not include 1 (Bablok & Passing 1985, Jensen & Kjelgaard-Hansen 2006. Individual electrolytes were analysed separately and for analysis of the Na + : K + ratio, the machine manufacturers' reference range of 27 to 40 was investigated first. Additionally, a narrower Na + : K + ratio range of 27 to 32 was also examined as it is defined as the ideal range in the public assessment report for the licensed formulation of desoxycorticosterone pivalate (DOCP) in Europe (European Medicines Agency 2019). This narrower range is the basis for dose adjustments of DOCP according to the data sheet of the licensed product.
Linear regression was performed to evaluate the effects of individual patients, age, sex, neuter status, treatment (fludrocortisone vs DOCP) and sample number. Other haematological and biochemical variables were not routinely recorded so could not be analysed. Potentially significant variables (P < 0.2) were carried forward into a generalised mixed linear model with significance set as P < 0.05. Additionally, analyser drift over time was also assessed by comparing the results from the first and last three dogs tested on each analyser by a two-way t test.
For both POC analysers, the sensitivity and specificity were calculated for detecting [Na + ], [K + ] and [Cl − ] outside their reference range relative to the ISE reference method (Table 1). This assumed that the ISE method was 100% sensitive and specific. Receiver-operator characteristic (ROC) curve analysis was also performed to allow comparison of area under the curves (AUC) using the Delong method. For analyses, P values <0.05 were considered statistically significant. As there were so few, these samples were treated independently for statistical analysis. Some dogs had only one blood sample tested while others had up to 18 samples included at various time points, measured between both analysers. There were 45 dogs tested on the Catalyst (median of eight samples per dog, range 1-14 samples) and 29 dogs tested on the VetStat (median of 1 sample per dog, range 1-4 samples).

RESULTS
Examination of the Bland-Altman plots revealed that the Catalyst tended to give higher results than the reference ISE method for [Na + ] and [Cl − ]. The [Na + ] median difference was slightly higher than the target (4.0 mmol/L) at 5.0 mmol/L, however, the LOA were well outside (−6.0 to 12.3 mmol/L). The [Cl − ] median difference was acceptable (2.5 mmol/L) but had wide LOA which were unacceptable (−2.5 to 9.6) ( Table 2, Fig 1). In contrast, the [K + ] had a median difference of just 0.2 mmol/L and narrow LOA (−0.9 to 0.7 mmol/L), however, due to the smaller range over which it is measured these differences may still be clinically important. This unreliability, especially of the [Na + ] also caused a wide 95% CI in the results of the Na + : K + ratio.
Cohen's Kappa coefficients showed agreement for [Na + ] was moderate at 0.51, [K + ] was moderate at 0.46 and [Cl − ] was fair at 0.33. The Na + : K + ratio agreement was good at 0.61 and when a narrower range of 27 to 32 was applied remained good at 0.63 (Table 3). No significant difference was found to suggest there was any analyser drift over time.
Passing-Bablok regression analysis identified constant and proportional bias with the Catalyst for [Na + ] and [Cl − ] but not for [K + ] or Na + : K + ratio (Appendix S1, Supplementary material). Compared to the ISE method, the Catalyst tended to under-estimate at lower sodium concentrations and overestimate at higher concentrations, while the reverse was true of chloride concentrations.
Examination of the Bland-Altman plots for the VetStat revealed that it also tended to give higher results than the reference ISE method for all analytes other that [K + ] ( Table 2, Fig 1). Sodium analysis had an unacceptable median difference of 11.5 mmol/L with wide LOA from −0.05 to 16.9 mmol/L. Similarly, [Cl − ] analysis showed an unacceptable median difference of 6.3 mmol/L with wide LOA of 1.2 to 12.9 mmol/L. Potassium analysis, however, was again better with a median difference of 0 mmol/L and narrow LOA of −1.4 to 0.6 mmol/L.
Cohen's Kappa coefficients showed agreement for [Na + ] was fair at 0.24, poor for [Cl − ] at 0.14 but very good for [K + ] at 0.82. The Na + : K + ratio was moderate at 0.60 but when a narrower range of 27 to 32 was applied it fell to 0.48 (Table 3, Supplementary material). No significant difference was found to suggest there was any analyser drift over time.
Passing-Bablok regression analysis identified proportional bias with the VetStat analyser for [Cl − ] and Na + : K + ratio and constant bias for [Cl − ] but no bias for [Na + ] and [K + ] (Appendix S1, Supplementary material). Compared to the ISE method the VetStat tended to have larger errors in [Cl − ] analysis at lower concentrations and smaller errors at higher concentrations while the opposite was true for [Na + ].
Both analysers failed to meet the precision targets for [Na + ], [K + ] and [Cl − ] with TE (Obs) higher than the TE A in all cases other than [Cl − ] measured on the VetStat which was just acceptable ( Table 4). The coefficient of variation (CV) for all analytes was relatively small on both analysers so this failure was largely due to the large bias that was present in both machines.
The clinical relevance of the disagreement between these methods was investigated by assessing how often the POC analysers produced results which fell outside their normal reference range when the ISE method found them to be within reference and vice versa. For the Catalyst, there were 21 cases (6%) with discordant [Na + ] results, 27 cases (8%) with discordant [K + ] results and 46 cases (14%) with discordant [Cl − ] results. The VetStat, meanwhile produced results which disagreed with the ISE method in 19 cases (26%) for [Na + ], 3 cases (4%) for [K + ] measurement and 9 cases (13%) for [Cl − ] analysis. When using a narrower "normal" range for the Na + : K + ratio of 27 to 32, the results were different in 47 cases (14%) from the Catalyst analyser compared to the reference laboratory. While with the VetStat, disagreement occurred in 14 cases (19%).
The sensitivity and specificity of the POC analysers was investigated with relation to their ability to detect electrolyte derangements outside of their respective reference intervals compared to the reference laboratory method ( Table 5). The Catalyst analy-

DISCUSSION
Our results indicate that the [Na + ], [K + ] and therefore the Na + : K + ratios, as well as the [Cl − ] measured by the Catalyst and VetStat analysers may not be used interchangeably with those from a reference laboratory analyser using an indirect ISE method. When analyser-specific reference ranges were applied to assess the agreement between analysers through the use of Cohen's Kappa coefficient analysis, [Cl − ] κ values were only fair or poor and [Na + ] analysis performed only slightly better, being classified as moderate or fair. On the other hand [K + ] analysis was more reliable and was classified as very good when measured on the VetStat analyser. Some variation was expected given the different methods involved; however, the magnitude especially of the difference in [Na + ], was clinically important for dogs treated for hypoadrenocorticism. Electrolyte derangements are common with many disease processes and it is important that POC analysers can accurately measure electrolytes across a range of concentrations. Dogs with low Na + : K + ratios are most commonly diagnosed with hypoadrenocorticism (Nielsen et al. 2008); however, low ratios also occur frequently with many other diseases, especially those affecting the urogenital, cardiorespiratory and gastrointestinal systems. Additionally, the data sheet for the formulation of DOCP (Zycortal 25 mg/mL, Dechra Ltd.) licensed for use in the European Union (European Medicines Agency 2019), relies on the use of Na + : K + ratios for dose adjustments. The Na + : K + ratios were often very different when produced on these POC analysers compared to those from the reference laboratory sometimes being higher and other times are being lower. Based on this analysis, there were many instances where treatment decisions would have been different depending on which analyser was used. This was also true using the sodium and potassium concentrations themselves when examined within the analysers reference ranges compared to the reference laboratory. This is clinically relevant as in up to 14% of samples run on the Catalyst and 26% of samples run on the VetStat, different and potentially deleterious treatment decisions may have been made. For the purposes of this study, that may have meant under or overdosing dogs with DOCP, but this is likely clinically relevant in conditions other than hypoadrenocorticism.
Previous studies have shown other POC analysers, utilising a different methodology (direct potentiometry with an ISE), to have poor agreement and correlation when measuring electrolytes (West et al. 2014, Uyanik et al. 2015; however, to the authors knowledge this is the first published study investigating the IDEXX Catalyst Dx for electrolyte analysis and the first to investigate electrolyte analysis on the IDEXX VetStat analyser to be published in a peer reviewed journal rather than by the manufacturer (IDEXX Laboratories 2006). POC analysers manufactured by IDEXX were the most commonly used (85%) in a survey of veterinary practices (Bell et al. 2014). More than two thirds (71%) of respondents also reported that they used the reference ranges supplied by the manufacturer without further adjustment or assessment.
Both analysers did not provide acceptable electrolytes results on repeated analysis according to the ASCVP TE A limits. No investigation of inter-assay variation was performed so it is possible that significant CV and bias also existed between runs. Both POC analysers had routine quality control tests performed regularly. It is likely that this was more often than would be performed in a first opinion practice given the setting in a busy teaching hospital.
Strictly speaking, the results of this study can only be directly applied to the individual Catalyst and VetStat analysers which were assessed at our hospital and do not necessarily apply to all analysers from this manufacturer, of the same model or of other analysers utilising ES and OF methods. A previous study has documented variation in the diagnostic performance between analysers of the same model as one of those used in this study (IDEXX Catalyst Dx) as well as many others not in the present study (Rishniw et al. 2012). An individual analyser's performance should therefore be evaluated, and the same analyser should ideally be used for all repeated analysis. This may be difficult when model specific reference ranges are produced by the manufacturer as with the Catalyst (IDEXX Laboratories 2015) and VetStat (IDEXX Laboratories 2010).

Limitations
Our study had several limitations. Firstly, given the population of dogs which were studied, there were no samples with a [Na + ] which fell above the upper reference range of the reference indirect ISE method. Therefore, it is possible that different biases may exist in this untested zone which were not identified in this study. This also limited the calculated sensitivity and specificity of the POC analysers. Secondly, only dogs with hypoadrenocorticism were included in the current study. While we feel it is unlikely, there could be an as yet unidentified substance (matrix effect) in dogs with hypoadrenocorticism which is not present in healthy dogs or dogs with other disease conditions which could have interfered with electrolyte analysis. Some caution should therefore be applied when extrapolating the results from this study to other canine diseases and especially to conditions in other species. Thirdly, this study took place over a long period of time and some change in analyser performance should be expected. This themselves rather than by the manufacturer (Bell et al. 2014).
There was no evidence of change over time from our data which may have been due to the regular servicing and quality control procedures within our hospital. Other biochemical and haematological parameters were not recorded during this study and it is possible that some of the dogs may have had altered PCV, protein or lipid levels. These physiological abnormalities could have impacted electrolyte analysis and varied between samples. However, there was no evidence of this in unpublished data from 30 of the included dogs that were also enrolled into a clinical trial. This also represents normal clinical practice and should have been accounted for to a degree by using paired samples. The effect of individual dog, age, sex, treatment and sample number were not investigated in this study. Some dogs were included up 9 British Small Animal Veterinary Association to multiple times while others were included only once on each analyser. These dogs may have had some unknown quality which affected analysis by one method and not another; however, this was felt to be unlikely and would not be accounted for in clinical practice. General linear mixed models did not identify any effect of individual dogs on the electrolyte results obtained. Additionally, the VetStat analyser testing was performed on heparinised whole blood rather than plasma as with the Catalyst and indirect ISE method. It is possible that this caused a difference in the analysis and future studies should look at any difference between electrolytes analysed in whole blood, plasma and serum on this analyser. It is well documented that whole blood and plasma samples invariably produce higher [K + ] compared to serum depending on platelet count (Schindler et al. 2018). Future studies should aim to assess electrolyte analysis in a range of other medical conditions, as well as in normal dogs over a wider range of [Na + ], [K + ], [Cl − ] and Na + : K + ratios. Additionally, different POC analysers using different methodologies (e.g. direct ISE) could be investigated as well as between day variations.
In conclusion, this study demonstrates the inherent differences between the electrolyte concentrations measured by these two POC analysers and a reference laboratory method in dogs with hypoadrenocorticism. As a result, it is suggested that the same analyser be used for all dose adjustments and repeated electrolyte analysis, with attention paid to the individual reference range for that machine.